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AjfAAj  machine  INTEGRATION  DESIGN  & ANALYSIS  SYSTEM 
(MIDAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 

PHASE  TV: 

OVERVIEW 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 

This  document  is  the  Software  Product  Specification  for  the  Man-Machine  Integration 
Design  and  Analysis  System  (MIDAS).  Introductory  descriptions  of  the  processing 

individual  modules  included  in  Annexes  A J. 

1.2  SCOPE  OF  DOCUMENT 

The  A3I  Program  is  a phased  development  program  in  which  software  development  talces 
olace  in  well-defined  cycles,  beginning  with  an  off-site  planning  meeting  to  c?t^_ 
i^u^mcnts,  progressing  trough  system  design  and  “XI 

.0  .he  co%rration,  and  ™^rTu,ed 

changes.  For  historical  infoimanon  regarding  Phases  I,  II  and  I^fte  rwder  is  urgeo 
consult  Section  9,  Historical  Information.  Documentation  from  previous  phases  is 
referenced  in  Section  2.2,  Information  Documents. 

Thic  document  is  intended  for  use  by  programmers  and  other  technical  specialists  working 
with  MIDAS.  Sufficient  high  level  information  is  provided  to  allow  any  reader 
familiar  with  the  objectives  of  the  A3I  Program,  the  current  (af.^  ^s  ^,0US)  ^ 

architecture  and  development  philosophies,  while  at  the  same  time  containing 
S?pleSemS  detail  useful  to  programmers  involved  with  specific  application  software 

modules. 

1.3  PURPOSE  AND  OBJECTIVE  OF  DOCUMENT 

The  ournose  of  this  document  is  to  meet  three  objectives:  one,  to  record  the  status  and 
phUosophy  of  design  of  MIDAS  as  it  existed  at  the  end  of  Phase  IV, 

EroSrLers  with  Efficient  implementation  detail  to  facilitate  u“°rt^X"/nted 
MIDAS  in  particular  application  environments,  and  three,  to  provide  twhnicdly  tmented 
ISs  wuHLnr  dtXled  description  of  MIDAS  rhan  can  be  given  rn  a typrcal  3-honr 
demonstration. 
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2.0  RELATED  DOCUMENTS 

2.1  APPLICABLE  DOCUMENTS 

Army-NASA  Aircrewl Aircraft  Integration  Program,  A3l,  Executive  Summary,  1 Sept, 

2.2  INFORMATION  DOCUMENTS 

«—  ■— > 

v$z 

-*2!S?C2^ 

SSSftae«». 

3.0  CONCEPT 

3.1  DEFINITION  OF  SOFTWARE 

MIDAS  is  an  integrated  suite  of  software  components  that  constitutes  a Dmrntvne 

in-the-loop  studies,  to  discover  problems  and  ask  "what  if'  questions  regarding  Hie 
projected  mission,  equipment,  and  environment.  4 g ing  the 

3.1.1  Purpose  and  Scope 

r^lwsgssssstsset^ 

Huma„  ofiiKSS™1  ^ "*  ^P-^onaJ 
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syTtets  3*2^^ 

results  are  presenieo  IF*P‘  „ J ' MIDAS  is  similar  in  concept  to  computational 

improve  designs  and  reduce  costs. 

j cian  MIDAS  gives  designers  an  opportunity  to  see  it  before  they  bui  a * 

and  principle  basis  permits  generalization  to  other  vehicles. 

3.1.2  Goals  and  Objectives 

Th*  a3t  Program  has  three  major  goals  organized  under  a research  and  development  umbrella 
Svelg^vSTrSJ oSJfrom  the  sponsoring  and  conduct  oftesicrescarehin 
perception  to  the  application  of  software  engineering  practices  to  provide  useable  human 
factors  tools.  These  goals  are: 

1)  Develop  an  integrated  methodology  for  crew  station  Proto^g  on 
models  and  principles  of  human  factors  and  engineering  psychology.  This 
methodology  shall  be  developed  around  a flexible  facility  for  collecting  and  us  g 
toolsAnodds  which  initially  £an  answer  specific  critical  questions  required  by  the 
existing  crew  station  design  process. 

Advance  the  capabilities  and  use  of  computational  representations  of  human 

in  .Knceptual  design,  synthesis,  and  analysis  of  manned  systems. 
These  representations  or  models  should  be  normadve  if  possible,  and  focused  for 
use  in  a simulation  with  explicit  inputs  and  outputs. 

3)  Transfer  the  relevant  technology/findings,  amassed  from  the  above,  to  interested 
research  and  practitioner  organizations  in  industry,  government,  and  academia.  The 
A3I  Program  (through  consortia)  must  lay  the  foundations  for  participant  by  the 
larger  community  of  researchers  and  designers  who  are  contnbuting  to  the 
development  of  computational  human  factors  or  who  might  become  users  of  the 
methods  and  models  developed  by  researchers  in  the  field. 

Three  goals  are  to  be  accomplished  by  developing  a Human-Factors  / Computer- Aided- 
Fngineerine  (HF/CAE)  system,  called  MIDAS,  which  assists  designers  m considering 

software  system,  MIDAS,  must  meet  the  following  objectives. 

1 1 MTD  AS  must  be  maintained  as  a flexible  integrated  modeling  environment.  An 

enheenuent  tools  based  on  them,  will  exist  in  various  formats  and  stages  ot  development. 
Consequently,  the  architecture  of  MIDAS  must  be  modular,  allowing  for  the  existence  of 


Page  3 


collection  of  models  rather  than  one  monolithic  model.  The  collection  of  tools  will  now  and 
change  over  time.  MIDAS  must  be  designed  as  a framework  within  which  a heterogeneous 
collection  of  tools  and  models  can  be  integrated  and  used  effectively  by  a design  team.  It  is 
also  intended  that  the  MIDAS  workstation  will  serve  a dual  purpose  by  providing  an 
integrated  environment  for  inspection  and  testing  of  new  models.  As  the  workstation 
architecture  evolves,  it  is  expected  that  both  deterministic  and  stochastic  constructs  will  be 
required  within  the  same  simulation  environment  to  make  use  of  the  widest  possible  range  of 
extant  models.  The  use  of  abstraction  is  also  strongly  encouraged  as  part  of  this  integrated 
environment.  The  goal  is  to  be  able  to  simplify  each  model  from  precise  computational 
representations  to  approximate,  qualitative  models,  as  determined  by  the  interactions  under 
study  and  the  answers  desired. 

2)  MIDAS  must  be  supportive  of  a wide  range  of  designer  activities.  As  described  in  a 
study  commissioned  by  the  A*I  Program  (Cody  1988),  the  crew  station  designer  engages  in  a 
wide  array  of  activities.  Initially,  the  tools  and  models  contained  within  the  MIDAS 
workstations  must  support  design  specification,  static  analysis  and  dynamic  analysis  growing 
lT  thex!^r*  complex  synthesis  task  once  successful  at  these.  To  support  the  specification 
phase,  MIDAS  must  contain  tools  to  allow  the  user  to  input  or  "specify"  the  elements  of  the 
mission,  crew,  cockpit,  vehicle  performance,  and  environment  that  are  given  or  known.  Where 
possible,  routine  static  analysis  tasks  such  as  assessing  reach  or  visibility  under  fixed  conditions 
must  also  be  supported.  However,  the  majority  of  the  difficult  and  interesting  crew  station 
design  problems  will  require  a dynamic  simulation  and  analysis  capability.  Only  through  this 
capability  will  users  be  able  to  discover  the  inherent  complex  interactions  among  the  operator 
task,  equipment,  and  environment  which  drive  aspects  of  workload,  performance,  and  training. 

3)  MIDAS  must  support  and  facilitate  interdisciplinary  communication.  Historically  a 

major  impediment  to  rational  cockpit  design  has  been  the  lack  of  communication  among  the 
practitioners  of  the  various  design  disciplines  involved  in  the  development  process.  Even 
when  placed  in  proximity  on  a development  team,  there  is  frequently  a lack  of  appreciation  by 
team  members  for  each  others  positions,  due  in  large  part  to  differences  in  training,  language 
(terminology),  objectives,  and  points  of  view.  Development  team  specialist  disciplines  may 
include  design  engineers,  avionics  engineers,  mission  specialists  (operations  and  doctrine) 
pilots,  engineering  psychologists,  training  systems  specialists,  reliability/maintainability 
analysts,  and  cost  and  production  experts.  MIDAS  must  attempt  to  provide  a basis  for 
communication  among  these  specialists,  integrating  the  information  generated  by  them, 
possibly  arbitrating  between  them,  alerting  users  to  the  need  for  specialists  not  cunently 
engaged,  and  mediating  between  the  design  team  and  the  end-product  user  or  sponsor. 

3.1.3  Description 

MUMS  consists  of  a set  of  software  components  providing  multiple  perspectives  or 
windows  which  contain  information  about  the  mission,  operator,  and  environment  in 
varying  levels  of  modeling  detail.  Visualization  is  emphasized,  with  3-D  graphic  and  iconic 
usenSentatl°nS  USe<^  ^ rna^°r  means  communication  to  the  wide  range  of  potential 


Figure  1 depicts  what  is  notionally  expected  to  be  contained  within  MIDAS  and  shows  how 
these  various  components  might  interact  and  overlap  to  address  the  various  stages  of  crew 
station  design  and  analysis.  6 
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Figure  1.  Functional  Content  of  MIDAS 

Forming  the  core  of  this  system  will  be  a number  of  human  behavior  and  pe^rmance 
models  intended  to  address  the  critical  areas  of  perception,  cognition,  workload,  etc.  as 
shown.’  Surrounding  and  augmenting  this  core  will  be  heuristic  methods  or  knowledge 
bases.  These  are  needed  to  supplement  what  is  explicitly  known  and  capable  of  being 
modeled  about  the  operator  in  order  to  complete  the  human  representaaonand  foirna 
closed-loon  system  Included  within  this  overall  funcdonal  context  will  be  a number  of 
SBSl  design  and  describe  elements  of  .he  cockpit,  envnonment.  and  htsk. 
used  as  stimuli  for  a range  of  models.  Finally,  analytical  methods  are  also  lmcKtded  lo 
address  anticipated  training  requirements,  summary  mission  results,  and  task  analysis 

parameters. 

These  notional  elements  are  shown  distributed  among  the  specification,  static  analysis,  and 
dynamic  analysis  stages,  indicating  a high  degree  of  overlap  between  theseP^^ 

Arrows  between  such  phases  are  meant  to  depict  that  the  process  is  not  constrained  to 
follow  a linear  progression  from  specification  to  static  analysis  and  then  dynamic  analysis, 
but  instead  coufd  be  used  to  begin  with  results  from  a particular  stage  and  ^en  progress 
backward  to  find  out  characteristics  of  the  crew  station  or  environment  which  are  the 
"drivers"  for  such  predictions. 
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3.2  USER  DEFINITION 

The  study  cited  above  (Cody  1988)  found  that  crew  system  design  involved  manipulating 
the  effects  of  six  major  system  factors  and  their  interactions.  These  factors  are  aircrew 
Cquipmenf  J°b  design,  aiding,  and  protection.  Furthermore, 
!^Xin,Und  thw  dCflgner?’  as  mdividual  problem  solvers,  engage  in  seven  major  classes  of 
activity,  problem  formulation,  synthesis,  consequence  finding,  consequence  analvsis 
fabncanon  and  prototyping,  eval  uation  and  judgement,  and  information  control 
Additionally,  the  study  found  that  approximately  50  distinct  disciplines,  ranging  from 
acoustics  to  probability  theory,  were  involved  in  the  crew  station  design  process 

maJ°r  st“dy.  Finally,  the  report  surveyed  existing  and  emerging 

fonms  of  design  support  and  found  that  less  than  one  third  of  the  design  problem  space 
(characterized  as  a matrix  of  the  aforementioned  activities  and  factors),  had  some  form  of 
tool  for  support.  More  significantly,  virtually  none  of  these  models  or  tools  surveyed 
sy^emfactorsC°mP  CX  intcractions  betwtcn  combinations  of  the  design  activities  and 

The  substantial  body  of  data  and  information  generated  during  the  simulation  must  be 
uiterpreted  in  a manner  that  is  meaningful  and  relevant  to  various  specialists  evaluating  a 
candidate  cockpit  design.  Because  the  crew  station  design  process  involves  professionals 

th^Wr^lde/ange  ?f  dfsclPhnes> the  A3I  Program  has  chosen  to  emphasize  visualization  as 
the  primary  form  of  information  output.  Graphic  and  iconic  representations  are 
predominantly  used  as  a means  to  foster  interdisciplinary  communication. 

Furthermore,  because  the  crew  station  design  process  includes  extremely  varied  activities 
TP°^els  Con,t?ined  Wlthl.n  the  workstation  are  designed  to  support  specification 
andJfa^  r?a  fS1S  tiwdl  as  dynamic  analysis  and  simulation.  For  example,  analysis  of 
reach  and  fit  of  cockpit  controls  can  be  performed  on  geometric  representations  of  the 

SSS  lfe  2***  datx?rale  vehicIe  dynamics/systems  or  human  performance 

simulation  models.  On  the  other  hand,  modeling  the  complex  interactions  of  competing 
uisks  during  the  performance  of  a mission  requires  considerable  dynamic  modeling  both 
human  behavior/performance  and  vehicle  systems  to  capture  the  context-sensitive,gtime 
varying  nature  of  task  sequencing  and  resultant  loads. 


3.3  CAPABILITIES  AND  CHARACTERISTICS 
The  components  of  MIDAS  at  the  end  of  Phase  IV  include: 


1)  The  Symbolic  Operator  Model,  which  contains  methods  to  1)  represent  and 
decompose  decomposition  of  mission  goals  to  their  lowest  level,  2)  sort  these  matched 

f^CtlVlt^PKtternS  by-  P/T'y-  3)  fi.nd  matching  equipment  operation  patterns  or 
activities  which  will  satisfy  them,  4)  interact  with  the  scheduling  and  loading  operator 
model  components  as  appropriate  and  5)  execute  these  activities  subject  to  physical 

JSSfKftjft  T’  et5  requirements,  Visual  Auditory,  Cognitive,  and  Psychomotor 
(VACP)  load  limits,  and  temporal/logical  constraints. 

. . The  Scheduler  (Z),  which  solves  for  a near-optimal  sequence  and  schedule 

based  on  a strategy  of  either  time  minimizing  or  load  balancing,  intended  to  represent 
possible  operator  behaviors.  v 


3)  The  Task  Loading  Model,  which,  in  accordance  with  current  research  in 
multiple  resource  theory,  classifies  individual  tasks  in  terms  of  their  demands  on  the 
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Visual  Auditory,  Cognitive  and  Motor  processing  dimensions  based  on  attributes  of  the 
mission  tasks,  world  state,  operator,  and  crew  station  equipment. 

41  The  Symbolic  Equipment  Models,  which  allow  characteristics  of  the  crew 
station  5m>mentto  be  Rented  in  terms  of  both  physical  and  functronal  attributes 
which  are  used  to  specialize  the  mission  tasks  prior  to  a simulation. 

5)  The  Visual  Editor  and  Simulation  Tool  (VEST),  which  provides  an ^^ctive 
3 D tool  used  to  create,  control  and  observe,  from  several  perspectives,  the  3-D  graphic 
envto^ofthSsion  simulation.  VEST  includes  the  The  Cockp.1  Design 
rrnFt  comnonent  containing  3-D  CAD  utilities  for  graphically  prototyping  the  crew 
£tion  geE? ! insTnUf.  conirols,  and  displays  using  a built-in  hbnuy  of  pnnuuve 

cockpit  objects. 

61  The  Display  Layout  Analysis  (DLA),  which  is  a prototypical  tool  intended  to 
provide  designers  with  assistance  in  determining  the  spatial  configuration  of  cockpit 
displays. 

7)  The  Anthropometric  Model  (Jack),  which  provides  realistic  and  physically 
quantifiable  human  figure  definition  and  motion  within  a 3-D  space  environment. 

81  The  Vision  Models,  which  are  fully  integrated  into  the  Jack  environment, 
include  Xe  Volume  Field-of- View  Model,  which  provides  computer  graphic  methods  for 
representing  the  relationship  between  two-dimensional  visual  fieid  maps  a^  the  three- 
Smension  J visual  space  they  serve,  and  the  Cockpit  Display  Visibility  Model  which 
provides  an  assessment  of  the  visibility  of  cockpit  objects  imaged  on  the  retina  in  terms  o 

a visual  system  footprint. 

9)  The  Aerodynamics  & Guidance  Model  (AGM),  which  represents  rather  generic 
helicopter  guidance  and  dynamics  for  uncoupled  controls. 

in\  -rue  simulation  Executive  and  Communications  Module,  which  facilitates  inter- 
machine coi^unication  and  dynamic  message  sharing  between  all  ^AS  ^mjwnents, 
using  a "data  pool"  concept  during  the  simulation  to  synchronize  and  distribute  state 
variables  among  the  simulation  processes  and  objects. 

1 1)  The  Training  Assessment  Module,  which  provides  a means  to 
training  media,  instructional  techniques,  and  time  necessary  to  qualify  in  the  cockpit  under 
Solment  This  module  was  developed  in  Phase  HI  and  not  modified  during  Phase  IV, 
detailed  documentation  of  this  module  is  included  in  the  Phase  III  design  document. 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 

One  tvoe  of  user  of  MIDAS,  a crew  station  designer,  might  first  approach  MIDAS  with  a 
requirement  ^design  anew  Crew  station.  He/she  would  have  a mission  that  must  be 
performed  and  fromthat  derive  a preliminary  specification  of  the  kinds  of  mformation  that 
must  be  presented  to  the  operator  in  the  crew  station.  The  initial  point  of  conract  wi* 
MIDAS  could  be  through  the  Display  Layout  Analysis  tool,  through  which  Ae  designer 
could  plan  and  evaluate  alternate  configurations  for  displays  in  thecrew  station.  With 

initial  configuration  in  mind,  the  designer  could  then  use  the  Cod^PU  designer 

render  a 3D  graphic  representation  of  the  displays  and  controls  in  the  cockpit,  pie  designer 
SSd  then  pm  Jack  iX  cockpit  and  perfor!n  some  static  analyses  of  reach  and  fit  using 
Zms SIS.  Physical  Characteristics.  Field  of  view  and  visibility  analyses  could 
be  performed  in  the  context  of  Jack  using  the  Vision  Models. 
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To  perform  a dynamic  analysis  of  the  interaction  of  a human  model  in  the  cockpit 

i!;,3  sc»"ai?01’  'Yith  Soa^s  and  activities  would  be  defined  in  the  context 

of  the  Symbolic  Operator  Model.  In  addition,  he/she  would  specify  both  the  physical  and 

iUSknlSCtenSdCS  0f  t5e  C0J*Pit  P ment  by  selecting  w Gilding  eqSpS 

models  With  the  mission  and  cockpit  environment  defined,  the  designer  could  run  a 
simulation  of  the  mission  scenario  and  observe  a 3D  graphic  rendition  of  operator  behavior 
1 w F°P°s*d  and  worId  environment.  Data  on  satisfied  and  unsatisfied  goals 

Sh  S f16  Srb°!iC  ?perat0r  modeh  Mission  mks  be  evaluated  by 
cogduve^?mot^  ‘°  °btajn  ,0ad  lnces  ^ the  four  dimensions:  visual,  auditory, 

4.0  REQUIREMENTS 

A number  of  key  developmental  aspects  must  be  mentioned  pries-  to  describing  Phase  IV 
specifics  becausethey  are  central  requirements  for  the  overall  A3I  Program.  Periiaps  most 

A thC  P?gn21  attempts  l°  use  extant  software  and  systems  to  the  greatest  extent 
possible  A considerable  amount  of  research  effort  and  money  has  been  expended 
nationally,  over  more  than  two  decades,  to  produce  analytic  methods,  models  and 
stnictures  representing  the  behavior  and  functions  of  human  operators,  avionics  systems 
“JL nis“°ns>  Wlt,h  vai7ing  degrees  of  success.  The  A3I  Program  is  not  expressly 
chartered  to  develop  new  models  and  methods.  Where  possible,  the  staff  will  selectively 
employ  those  which  have  already  been  developed  and  will  engage  in  research  and 
d^efJ°P,menl  of  new  models/methods  only  when  a critical  void  is  encountered.  The  advice 
of  the  Natl0nai  Research  Council's  Committee  on  Human  Factors  (offered  by 
die  study  group  on  Human  Performance  Models  and  the  report  Human  Performance  * 
Models  for  Computer-Aided  Engineering)  has  been  solicited  and  followed  on  this  matter. 

An  enormous  amount  of  modeling  is  anticipated  throughout  the  life  of  the  Program,  with 
the  operator,  vehicle  and  world  as  major  categories  of  models.  The  A3I  Program  has 

a simulatlon-based  approach  to  system  design  and  evaluation  that  proposes  to  make 
mSc  Im?6  °fgr1Phlu  311(3  Icomc  representations  of  the  underlying  model  structures. 

generally  be  prescriptive,  providing  results  relevant  to  mission  success  in 
terms  of  such  parameters  as  performance,  errors,  duration,  and  rates.  Wherever  practical 
cEan??iU(lS°deiS  WUh  -5' !°, years  ?f  development  and  validation,  supported  by  a 

^y, of  emPLnca]_data’  wil1  be  Utilized.  However,  many  instances  will  require 

fas  vet  to  fr T °ngT8  rcsearcb- ,wherc  final  verification  and  valkbtion 

as  yet  to  be  compIeted,  as  well  as  qualitative  models  emerging  from  the  AI  field  It  is 

expecied  that  the  MIDAS  workstation  will  sew  a dual  pu^osj  in  aiding  this  mocess  by 
providing  an  integrated  environment  for  model  inspection  and  testing.  g P y 

4.1  REQUIREMENTS  APPROACH  AND  TRADEOFFS 

On  the  first  day  of  the  A3I  Program,  what  needed  to  be  done  and  why,  was  known  at  least 
in  general  terms.  Howto  do  it  was  not  known,  nor  could  one  predict  very  far  into  the 
future  as  to  progress  which  might  be  expected.  The  inability  to  clearly  specify  the  ultimate 
unimlr  thetatedthe  adoption  of  certain  approaches  to  ensure  that  even  indie  face  of 
uncertainty,  this  highly  ambinous  program  could  be  directed  toward  its  goals  while 

users  and  coHaborators™18  imermediaK  producls  could  * mailable  io  interested 
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4.1.1  Incremental  Development 

In  the  A3I  Program,  a standard  practice,  incremental  development,  has  evolved  to  take  die 
following  programmatic  form.  Development  is  accomplished  in  phases  with  each , phase 
consS  canning,  design,  development,  testing,  demonstration  and  documentation  of 
SXe ‘to  fulfill  a s^cific  set  of  requirements.  A development  phase  begins^th  an  off- 
site meeting,  lasting  three  to  four  days  and  attended  by  the  full  in-house  staff,  extramural 
asscSates  tid  invited  peer  reviewers,  which  is  held  at  a location  conducive  to 
iS^rroptedddiberations.  During  the  off-site,  plans  are  made  for  the  next  phase  of 
development  based  on  what  has  been  learned  dunng  the  previous  phase,  «lvice  from 
reviewers  and  associates,  assessment  of  weaknesses  in  current  methods,  and ' 
comments/recommendations  made  by  visitors  and  reviewers  of  the  Prc^^df.  e p 
ohase  The  products  of  the  off-site  are  a formal  statement  of  requirements  for  the  next 
nhase  of  devefopment,  an  agreement  as  to  the  length  of  the  development  penod  (usually  8 
fo  14  monK ‘ S an  undemanding  among  the  staff  and  extramural  members  of  what  is 
expected  to  be  accomplished  — a common  vision. 

UDon  our  return  to  NASA-Ames  and  the  laboratory,  the  details  of  die  work  are  finalized 
and  development  begins.  Work  breakdown  structures  and  schedules  are  develop^  to 
track  phased  development  in  each  major  work  area.  Depending  on  the  length  of  the 
S.tS.pSa  number  of 

development  and  discover/correct  problems.  Two  weeks  befirc  *= md 
development,  the  writing  of  any  new  code  is  terminated.  Final  checks  ol  the  code  ana 

models  are  made  and  any  bugs  corrected. 

The  end  of  the  development  phase  is  marked  by  a series  of  demonstrations  highlighting  the 

hours  Depending  on  the  audience,  the  detail  of  the  demonstrations  may  go  down  to  the 
Sevel  Cch  is  leLed  during  these  demonstrations  from  the  visitors  to  the 
laboratory  Many  are  from  academia  in  the  fields  of  psychology,  AI,  aM  ComputCT  Stnence 
and  their^uggestions  and  comments  are  extremely  helpful,  frequently  being  used  as  tnpu 
for  the  next  phase  of  development. 

At  the  end  of  the  demonstration  period,  effort  is  devoted  to  documentation  (pnnwdy  this 
Software  Detailed  Design  Document)  and  recording  of  lessons  learned  in  preparation  fw 
fhfneSoM^ieetinl  By  this  method,  the  programhas  been  W-s^PP”*  ltself 
along  very  successfully  since  the  first  off-site  meeting  in  the  Fall  of  1985. 


Page  9 


Figure  2 below  depicts  major  phase  milestones  since  the  Program’s  inception. 
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PHASE  I 

Review,  Planning,  Doc 

PHASE  II 


Planning;  Acquire  Equipment  & Staff 
Proof  of  Concept  w/  Prototype  Mlsalon 


Tools  and  Architecture 
Development 


Review,  Planning,  Documentation 

PHASE  III 

Review,  Planning,  Documentation,  Tech  Xfer 

PHASE  IV 

Review,  Planning,  Documentation 

PHASE  V 


Workstation  Architecture 
& Modeling  Expansion 


Tool  & Model  Refinement 
Apply  to  New  Cockpit  Dev. 


T 


]Archft*ctur» 
Development 
Emphasis  on 
Simulation 


Figure  2.  A3I  Program  Timeline 


4.1.2  Development  Techniques 

is  a"etworked  collection  of  graphical,  numeric,  and  symbolic  processors  which 
forms  a flexible,  integrated  modeling  environment.  One  of  the  central  challenges  is  to 
design  and  implement  a general  environment  that  can  easily  accommodate  models  written  in 
various  languages  by  a variety  of  developers,  while  maintaining  inspectabili^and 

h^nU,S  at  VfnrOUS.levels  of  modeling  abstraction.  A number  of  methodologies  have 
been  used  to  satisfy  these  requirements.  6 

4. 1.2.1  Rapid  Prototyping 

inception,  the  A3I  Program  has  emphasized  the  evaluation  of  architectural  issues 
and  model  requirements  through  rapid  prototyping.  Much  of  what  is  being  attempted  by 
the  software  development  staff  is  radically  new  with  no  known,  proven  methods  Rapid 
prototyping  as  a means  to  develop  the  various  tools  and  models  has  both  guided  and  V 
acilitated  subsequent  effort,  providing  the  opportunity  for  scientific  scrutiny  and  user- 
community  feedback  before  an  overwhelming  development  effort  is  expended. 
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4 1 2.2  Distributed,  Tick-Based  Simulation 

PXnlet^ 

MIDAS  contains  no  requirement  for  a real-time  simulation  cap  ty. 

4.1.2.3  Graphic  & Iconic  Interface 

Visualization  methods  ^v"is^izMion^^ries^CTiM^on!wl 

compurntional  results  will  facilitate  the  use  of  thesystem  m a 

aS5SSEi£?S^Ssa-*Bt: 

form  which  is  meaningful  to  a wider  range  of  potential  users. 

4.2  HARDWARE  ENVIRONMENT 

The  A3I  Program  has  adopted  the  requirement  to  use  existing  and  proven ^ardware, 
IS^SSSd  Silicon Graphics 

hardware  environment  throughout  the  development  phases  is  described  in  Section  , 
Historical  Information. 
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Figure  3.  Phase  IV  Hardware  Configuration 


4.2.1  Symbolics  Lisp  Machines 

Model  3675  Color  Workstation  (Barracuda)  consisting  of: 

Monochrome  Console  with  OCLI  filter 

Keyboard  & Mouse 

45  MB  1/4"  Cartridge  Tape  Drive 

Ethernet  Controller  and  Transceiver 

22.5  MB  RAM 

Enhanced  Performance  Option 

338  MB  Fujitsu  Eagle  Disk 

550  MB  CDC  Disk 

Model  CG70-FB02  High  Resolution,  24-bits/Pixel  Color  Frame  Buffer 

Tektronix  19  Color  RGB  Monitor 

Model  OP36-FPA1  Floating  Point  Accelerator 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 

Symbolics  # STCP-1  TCP/IP  Software 

S-Group  (S-Paint,  S-Geometry,  S-Render,  S-Dynamics,  and  color  6 0 V405  13) 
Genera  7.2 

Model  3640  Color  Workstation  (Puffer)  consisting  of: 

Monochrome  Console  with  OCU  Filter 

Keyboard  & Mouse 

45  MB  1/4”  Cartridge  Tape  Drive 

Ethernet  Controller  and  Transceiver 

11.25  MB  RAM 

2-140  MB  Disks 

CAD  Buffer 

Tektronix  19”  Color  RGB  Monitor 
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Symbolics  # SLAN-FORT  Fortran  77  Compiler 

Symbolics  # STCP-1  TCP/IP  Software  *n  van*.  131 

S-Group  (S-Paint,  S-Geometry,  S-Render.  S-Dynamics,  and  color  6.0  V405.13) 

Genera  7.2 

Model  3640  Monochrome  Workstation  (Squid)  consisting  of: 


Monochrome  Console  with  OCLI  Filter 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
13.5  MB  RAM 
2-140  MB  Disks 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 
Genera  7.2 

Automated  Reasoning  Tool  (ART)  Version  3.2 

Model  3620  Monochrome  Workstation  (Sea  Slug)  consisting  of. 

Monochrome  Console  with  OCLI  Filter 

Keyboard  & Mouse 

Ethernet  Controller  and  Transceiver 

18  MB  RAM 

190  MB  ST506  Disk 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 
Genera  7.2 


4.2.2  Maclvory  Workstation 

Apple  Macintosh  IIx  Workstation  with  Symbolics  Maclvory  board  (Otter)  consisting  of 

19"  Radius  Monochrome  Monitor 
Keyboard  & Mouse 

Ethernet  Interface  card  ^ . ,, 

21  MB  RAM  (5MB  SIMM  CPU  memory  4-  16MB  Nubus  memory  card) 

300  MB  External  Disk 
Internal  3.5M  floppy  disk  drive 
40  MB  mini  cartridge  tape  drive 
Maclvory  floating  point  accelerator 
Maclvory  7.4 

Macintosh  system  software  version  6.0.2 
Microsoft  Word  4.0 
Mac  Draw  II  1.1 


4.2.3  Silicon  Graphics  Computers 

W-2500A  Workstation  (Orca)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  & Mouse 
45  MB  1/4"  Cartridge  Tape  Drive 
Ethernet  Controller  and  Transceiver 
12  MB  RAM 

2 474  MB  Fujitsu  10.5”  Disk  Drives 
HU-T04  Turbo  Option  W/4MB  RAM 
H3-FPA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z Clipping  Assy 
C-WTCP  IP/TCP  Software 
P-DBX  Dial/Button  Box 
Unix  System  V with  BSD  4.2 
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NFS 

C Compiler 

IRIS  Graphics  Library  II  and  Window  Manager 

W-3120  Workstation  (Manta)  consisting  of: 

19”  High  Resolution  Monitor 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
8 MB  RAM 

72  MB  Winchester  Disk  Drive 
H3-FPA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z Clipping  Assy 
C-WTCP  IP/TCP  Software 
P-DBX  Dial/Button  Box 
Unix  System  V with  BSD  4.2 
NFS 

C Compiler 

IRIS  Graphics  Library  II  and  Window  Manager 

W-4D120GTX  PowerSeries  Workstation  (Coral)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
32  MB  RAM 

380  MB  ESDI  Winchester  Disk  Drive 

Double  Buffered  1280x1024x4  Display  Memory 

Double  Buffered  Alpha 

24  bit  Z buffer 

C-WTCP  IP/TCP  Software 

P-DBX  Dial/Button  Box 

IRIX  System  V release  4D1-3.1D 

NFS 

C Compiler 
C++  Translator 
Fortran  77  Compiler 

IRIS  Graphics  Library  II  and  4Sight  Windowing  System 

W-4D220GTX  PowerSeries  Workstation  (Starfish)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
32  MB  RAM 

2 780  MB  ESDI  Winchester  Disk  Drives 

Double  Buffered  1280x1024x4  Display  Memory 

24  bit  Z buffer 

C-WTCP  IP/TCP  Software 

IRIX  System  V release  4D1-3.1D 

NFS 

C Compiler 
C++ Translator 
Fortran  77  Compiler 

IRIS  Graphics  Library  II  and  4Sight  Windowing  System 

W-4D20G  Personal  IRIS  Workstation  (Urchin)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
8 MB  RAM 

170  MB  SCSI  Winchester  Disk  Drive 


Page  14 


1280x1024x4  Display  Memory 
Double  Buffered  Alpha 
C-WTCP  IP/TCP  Software 
DUX  System  V release  4D1-3.1D 
NFS 

m?s°Graphics  Library  II,  4Sight  Windowing  System,  and  Environment  Manager 


4.2.4  Networking  Hardware 

CableTron  MT-800  Ethemet/IEEE  802.3  Transceiver 

4.2.5  Peripherals 

Okidata  Model  2410  Dot  Matrix  Printer 

Seikla  Lismmen^D^ScarfcHwIr  Color  Printer  & Multiplexor 
4.3  SOFTWARE  ENVIRONMENT 

The  A3I  Program  office  made  a decision  to  standardize  on  software,  such  as  system  software 

Sags^^ 


Svmhnlic  Models/Tools  Numeric  & Graphic  Mpdcls/TQfilS 


Operating  Systems 
Languages 

Object-oriented  methods 
Machine  Class 


Genera 
Common  Lisp 
Flavors 
Symbolics 


UNIX 

FORTRAN,  C 
C++ 

Silicon  Graphics  IRIS 


In  addition  to  the  software/OS  packages  described  above,  two  key  software  development 
approaches  support  the  A3I  Program  goal  of  providing  ^exibleenvironment  in  which 
, models  of  system  and  human  performance  can  be  integrated. 


various  i 


4.3.1  Object-Oriented  Programming 

The  A3I  Proeram  has  adopted  the  use  of  object-oriented  programming  methods,  where 
^ctfci^S rffon  to  manage  the  complexity  of  the 

modularity  and  abstraction,  as  well  as  to  promote  graceful  mcrei^ntal  srfw^e  ^ _ 

development.  While  not  universally  applied  to  every  component,  the  1 

paradigm  is  a natural  match  to  the  structure  of  the  man-machine  mtegration  proWems^^ 
investigated.  Object-oriented  techniques  support  extensive  reuse  of  software  structures  and 
are  an  evolving  standard  in  the  software  development  community.  Additionally,  use  o 

may  mduce  the  ultimate  code  bulk  by  matang  ex.ettstve  muse  of 

software  structures  and  can  be  written  to  be  virtually  self-documenti  g. 

4.3.2  Source  Code  Control 

Software  for  which  the  A3I  Program  does  not  have  access  to  source  code  (with 
mSSta  rights)  U generally  not  permitted  in  the  oonftgurauon.  Encumbrances  to 
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success,  since  technology  transfer  is  a primary  concern.  However,  to  date,  some 
exceptions  have  been  allowed  for  packages  with  clearly  defined  and  easily  accessible 
software  interfaces  that  would  be  inappropriate  for  the  Program  to  develop,  or  software 
development  utilities  (shells,  debuggers,  process  performance  metering,  function  libraries 
etc)  that  offer  superior  performance  which  other  organizations  have  ready  access  to. 
Exceptions  to  the  desire  to  avoid  encumbrances  are  two  software  packages  used  in  Phase 
l y > MultiGen™,  a commercial  3D  modeling  package  with  a hefty  license  fee  and  the 
Generic  Expert  System  Tool,  (GEST),  a package  with  a moderate  but  not  inconsequential 
license  fee  produced  by  Georgia  Technical  Research  Institute  (GTRI). 

*c  des'gn  and  analysis  tools  were  built  using  C,  Unix  System  V with 

BSD  4.3  extension,  and  the  IRIS  Window  Manager/Graphics  Library  n.  The  MultiGen™ 
modeling  package  was  used  as  the  undeipinning  for  the  CDE  and  VEST  components  as 
well  as  a visualization  medium  for  the  Aerodynamics  & Guidance  Module.  Most  Lisp  tools 
and  models  were  built  using  Symbolics  Common  Lisp,  with  the  Flavors  extension,  under 
Genera  7.2.  The  S-Packages  were  available  for  use  in  displays,  although  not  heavily  relied 
upon  during  this  phase.  Communication  software  used  the  TCP/IP  protocols  to 
communicate  between  the  Silicon  Graphics  and  Symbolics  workstations  over  Ethernet 


Two  expert  system  shells  were  introduced  in  Phase  IV  to  facilitate  development  of 
knowledge-based  approaches  in  certain  modules.  The  Generic  Expert  System  Tool 
(GEST),  produced  by  Georgia  Technical  Research  Institute  (GTRI),  was  used  as  the  basis 
for  the  Scheduler.  This  shell  includes  a blackboard  architecture,  which  facilitated 
development  of  this  module.  The  C Language  Information  Processing  System  (CLIPS) 
deve  oped  by  NAS  A-Johnson  Space  Center,  was  used  for  the  rule-based  portion  of  the  ’ 
Dispiay  Layout  Analysis  tool.  Source  code  is  available  for  both  of  these  shells  and  both  arc 
available  at  relauvdy  modest  cost  to  users  of  MIDAS,  although  GEST  is  more  expensive 
than  CLIPS. 


The  distribution  of  the  Phase  IV  models,  tools,  and  displays  across  the  available 
workstations  is  shown  in  Figure  4 below. 
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Ficure  4.  Distribution  of  Phase  IV  Software  Components  and  Displays 

within  the  A3I  Lab 

4.4  EXTERNAL  INTERFACE  REQUIREMENTS 

4.4.1  User  Interfaces 

The  user's  interface  to  the  computational  tools  and  models  of  MIDAS  is  extremely 
important  However  since  the  Program's  charter  is  to  develop  prototype  facilities  in  an 
architecture  that  is  continually  evolving  and  may  or  may 

delivery  platforms,  a polished,  consistent  interface  across  all  of  the  applications  has  not 
been  developed  or  considered  a priority  in  Phase  IV.  As  an  interim .solution,  each 
ESfcSS Svelo^er  was  encouraged  to  use  pop-up  windows  P^own  menus  ^d  the 
manipulation  of  graphic  or  iconic  representations  as  the  principle  mechanisms  of  user 

interactions. 

A hi  ah  priority  in  Phase  V will  be  to  design  a unified,  principle-based I approach  to  the 
implementation  of  user  interfaces  that  is  based  on  a thorough  study  of  user  r^uirements 
well  as  human-computer  interactions  (HCI)  research  (previous  and  ongoing)  at  various 
universities  and  industry  centers. 

4.4.2  Integration 

A considerable  degree  of  integration  among  the  various  modules  of  MIDAS  was  achieved 
in  Phase  IV  The  data  flow  within  MIDAS  during  Phase  IV  is  shown  in  detail  using  an  N2 
fiX, Riura 5 ^e  components  involved  are  portrayed  in  the  shaded  boxes  located  on 
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the  diagonal  of  the  figure.  As  shown  in  the  legend,  inputs  are  on  the  vertical  axes  of  each 
box  and  outputs  on  the  horizontal  axes. 
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Figure  5.  MIDAS  Phase  IV  Integration  N*  Chart 


For  example,  during  each  simulation  "tick",  the  Vehicle  Guidance  Model  computes  the 
demands  for  control  movement  based  on  the  current  vehicle  position,  orientation,  and 
speed  (from  the  Dynamics  Model)  together  with  the  next  desired  waypoint  location 
airspeed,  and  altitude  (from  the  Mission  Editor).  These  control  requirements  are  then 
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Integration  between  components  is  aducved  in  M<SfeV  tadUthere  are 

a-^FSS«« 


Jack  reach  command: 


Type  of  reach  activity. 
Reach  site  name. 


Position  and  availability  of  pilot's  left  and  right  hands  and  head  orientation: 


Left  hand  position  (x,  y,  z), 

A flag  indicating  the  availability  of  the  left  hand, 

Right  hand  position  (x,  y,  z), 

A flag  indicating  the  availability  of  the  nght  hand. 

Head  orientation  (yaw,  pitch). 

Ownship’s  position,  orientation,  and  important  aerodynamic  parameters: 


Position  (x,  y,  z). 

Orientation  (yaw,  pitch,  roll). 
Altitude  above  ground  level, 
Z component  of  velocity. 
Airspeed, 

Heading, 

Tumrate. 


Convoy  vehicle's  position  and  orientation. 


Position  (x,  y,  z), 

Orientation  (yaw,  pitch,  roll). 

Waypoint: 

Waypoint  ID, 

Position  (x,  y,  z). 

Multi-Function  Display  navigation  page: 

Format  (Centered  or  Decentered), 
Scale. 

Multi-Function  Display  aircraft  page: 
Torque, 

Specific  fuel  range. 

Endurance, 

Aft  tank  fuel  quantity, 


Page  19 


Forward  tank  fuel  quantity, 
Total  fuel  quantity, 

Engine  1 fuel  flow, 

Engine  2 fuel  flow. 

Total  fuel  flow. 


Multi-Function  Display  communication  page: 


Challenge  code, 

Reply  code, 

Frequency  Display  Buffer. 

4.5  REQUIREMENTS  SPECIFICATION 
4.5.1  Process  and  Data  Requirements 

tJ1C  d5veI?pmeni work  1,1  Phasc  IV  was  to  improve  the  breadth  and  depth  of 
existing  modules,  develop  and  integrate  new  components  and  achieve  a higher  degree  of 

£So°n  311,0118  UlC  Vari°US  comP°nents  t0  implement  a dynamic  simulation  of  Emission 

By  the  end  of  Phase  IV,  the  requirements  for  MIDAS  had  resulted  in  development,  both  in- 
torough^an^confract,  of  the  following  software  components,  hosted  on 
networked  Silicon  Graphics  IRIS  and  Symbolics  computers: 

1)  The  Symbolic  Modeling  component,  written  in  Symbolics  Common  Usd 
contains  methods  to  represent  and  decompose  mission  goals  to  their  lowest  level  match 
them  to  equipment-dictated  activities,  interact  with  the  scheduling  and  task  loading 
components,  and  execute  activities  subject  to  resource  constraints. 

Expert  S vsSS  T^ffCE^T^ ^rn^TPT^1^^0^  architecture  available  in  the  Generic 
zW? S®  P? 1 \GEST>  from  GTRI,  to  implement  a constraint-based  scheduler  using 
two  alternate  scheduling  strategies,  time  minimization  and  load  balancing.  8 

th  - . T,he  Task  V?adtog  AMcPel>  written  in  Common  Usp,  classifies  tasks  accoiding  to 
then-  demands  on  the  Visual,  Auditoiy,  Cognitive  and  Motor  processing  dimensions  and8 
passes  those  values  of  operator  loading  to  the  Symbolic  Operator  Model  to  establish 
resource  constraints  and  to  the  scheduler  for  use  in  scheduling  tasks. 

t.  *}. 1116  Symbolic  Equipment  Model,  written  in  Symbolics  Common  Lisp  models 
the  cockpit  equipment  as  finite  state  machines  to  represent  the  detailed  physical  and 
functional  attributes  and  to  specify  the  operation  of  equipment  during  the  simulation. 

„ 5)  The  Visual  Editor  and  Simulation  Tool  (VEST),  written  in  C as  an  extension  to 

T1?1"8 paikage>  MtotiGen™,  provides  an  interactive  3-D  trolSo 
create,  control  and  observe,  from  several  perspectives,  the  3-D  graphic  environment  of  the 

DmSitinTS"'  Cockpi.t  De?ign  ^‘tor  (CDE ),  a subsystem  of  VEST,  contains  3- 
bhQeS  £r  Prototyping  the  cockpit  geometry,  instruments,  controls,  and 

& display^  C3n  **  madC  l°  0thCr  m0dclS  °r  data  fi,es  for  atomation  of  selected  controls 

e "P^-^Ptoy  Layout  Analysis  (DLA)  tool,  written  in  C and  also  using  the  Expert 

System  shell  CLIPS,  developed  by  NASA,  is  a prototypical  tool  intended  to  provide 
designers  with  assistance  in  determining  the  spatial  configuration  of  cockpit  displays. 
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Badler  at  the  University  of  Pennsylvania. 

< />  it  * - — *- — * Tsp.k  environment 


at  uiv  — — - * 

8)  The  Vision  Models,  iSSSS 

f„ IS  Srdfe^th  Cel  developed  by  *e  I*s.  1.  Bergin 

and  J.  Lubin  of  the  SRI/David  Samoff  Research  Center. 

9)  The  Aerodynamics  & Guidance  nwd^d  in-housc.  It 

developed  by  Dr  Anil  Phatak  of  Ana  ytic  „ helicopter  providing  pitch,  roll,  yaw,  and 
contains  rather  generic  equations  o decounled  cycijc  collective,  and  pedal  movements. 
Stoe  muto  sp«d  and  horizontal/vertical  pad.  condol  feedback  as 

a simple  representation  of  piloting  activiues. 

10)  The Communicadons Module, written* bo* CajlU* £™£^Se 

cool  through  which  the  simulation  components  share  state  variables 

SS-bSed  simulation  by  sending  ticks  to  the  other  modules. 

1 1)  The  Training  Assessment: module, m°SSt^the  training 
Automated  Reasoning  Tool  (ART)  c e Pessary  to  train  various  operators  to  "initial 
media,  instructional  techniques,  ^n^aracterScs  This  component  was  developed  in 

included  in  the  Phase  III  design  document. 

5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

Figure  6 below  shows  the  components  of  M^AS  at entsof ^©AS  and  grounded  boxes 
relate  to  one  another.  'Hie  shaded  toxes  between  the  modules.  The  Mission  Editor  and 
show  the  nature  of  the  information  g or,mnnnents  of  the  Symbolic  Operator  Model 
Task  Representation  modules  are  “^S^TS^wckpit environment  created  with 
at  the  present  time.  Jack  integrated  cockpit  with  the  DMA  terrain  visible 

the  CDE  to  represent  the  pilot  and  CPG  in  their  coc  p simulation,  nor  is 

toough  the  windscreen.  The  Vision  Models  a^e  ^ ^ 

the  Display  Layout  Analysis  tool  or  the  although  the 

Task  Loading  Model  are  not  ae^ly  integr  Symbolic  Operator  Model  bypassed 

- Schcduto  wete  designed  to  perform  during 

the  simulation. 
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Figure  6.  MIDAS  Phase  IV  Top  Level  Software  Architecture 
5.1.1  Phase  IV  Development  Summary 

Responses  to  the  Phase  III  demonstrations,  as  well  as  discussion  at  the  Phase  IV  off-site 

nced  no*  only to  continue  the  development  of  the  core  set  of  A3I  models  and  tools 
£L/«Uy  rgr?tC  ?5m  into  * desiSners  workstation.  Also,  emphasis  would  have  to  be 
^n'l01}  expi-“1?'  addressing  h.ow  Sl)ch  models  and  tools  would  be  sensitive  to  cockpit  design 

thCre  ^aS  •nttrest  ,n  determini"g  the  potential  ramifications  of  hi  traducing 
the  glass  cockpit  concept,  that  is  the  use  of  Multi-Function  Displays  (MFD)  in  place  of 

« 3yS  m thC  co<rkPlt-  Accordingly,  the  focus  of  the  phase  was  to  use  a portion  of  a 
typicai  mission  segment  as  the  setting  for  a comparison  of  the  Apache  AH-64A  cockpit  with 

^ and  ?C  nCW  APache  Longbow  model,  in  which  many  functions  currently 
performed  with  dedicated  equipment  are  incorporated  into  Multi-Function  Displays  (MFD).  The 
intention  was  to  show  how  MIDAS  could  be  used  to  compare  operator  performance  in  the  same 
scena{1®  W1.th  two  different  cockpit  designs.  These  objectives  required  a degree  of 
and  detad  Previously  impossible,  and  drove  the  design  requirements  for  the  individual 
MIDAS  components  The  major  components  of  MIDAS  in  Phase  IV  are  described  briefly 

Annexes  A ,i,ro“gh  j which  comain  morc  *** 

5.1. 1.1  Symbolic  Operator  Model 

This  model,  coded  in  Symbolics  Common  Lisp,  contains  the  data  structures  and  methods 
used  to  represent  and  decompose  the  required  mission,  environment  and,  human 
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performance  models  of  the  crew.  This  component  ocpresse: * mssion  J^vWes  in  ^rms  of 
£nals  or  states  to  be  achieved,  allowing  the  user  to  explicitly  allocate  such  tasks  to 
eauiDmenuw5 human  operators,  as  well  as  providing  for  event-related  operator  responses 
which  cannot  cleanly  be  represented  in  an  hierarchical  fashion.  TJie  focus  for  the  Symbohc 
ModeUngcomponent  during  this  phase  was  on  the  design  and  coding  of  a generalizable 
framework  for  symbolically  representing  the  functions  of  cockpit  equipment  used  ° 
accomplish  mission  tasks.  This  framework  allows  various  cockpit  alternatives  to  be 
evaluated  without  completely  re-editing  the  mission  decomposition,  since  Redesign 
maintains  a distinction  between  the  physical  structure  (or  state  operators)  of  the  equipment, 
and  the  functional  requirements  (or  inferred  goals)  required  by  the  task.  frevious  A I , 
Kmbolic  models  ofthe  mission  and  pilot  tasks  failed  to  explicitly  depict  the  relationship  o 
theequipment  design  and  the  primitive  task  actions,  loading  values,  or  timelines. 

This  comDonent  is  understandably  one  of  the  most  complex  and  continually  evolving.  It 
^endy«s  two  major  subcomponents,  the  scheduling  and  loading  models  descnbed 
bSow^  During  a simulation,  this  model  attempts  to  execute  assigned  mission  acuvines 
subject  to  specified  constraints,  state  variables,  and  other  simulation  object  r^u^ements^ 
This  model  accomplishes  this  action  by:  1)  updating  the  simulated  operators  goal 1 
delete  terminated  or  inappropriate  goals,  2)  examining b 
to  determine  if  event-response  activities  are  required,  3)  tracing  the  d«co^P“1t°°n 
mission  goals  to  their  lowest  level,  finding  matching  equipment  operation  patterns  or 
activities* which  will  satisfy  them,  4)  sorting  these  matched  goal-activity  patterns  by 
priority,  5)  interacting  with  the  scheduling  and  loading  operator  model  components  as 
appropriate  (although  during  the  demonstrations,  this  interaction  was  not  performed)  and 
6? ?exSng  these  activitief  subject  to  physical  resource  (hand,  eye,  etc)  requirement 
Visual  Audhory,  Cognitive,  and  Psychomotor  (VACP)  load  limits,  and  temporal/logical 

constraints. 

Mission  task  environment,  or  operator  objects  are  instantiated  when  their  conditions  are 
met  executing  their  assigned  procedures  and  spawning  new  activities.  Contained  within 
the  various  task  objects  is  information  on  temporal  relationships,  preconditions,  logical 
consSs,  loading,  subtasks,  and  relative  priority.  In  this  manner,  the  decomposed 
n^ssion  serves  as  a forcing  function,  "driving"  the  interaction  of  various  models  used 
during  a simulation.  Refer  to  Annex  A for  a more  detailed  description. 

5.1. 1.1.1  Scheduler  (Z) 

This  constraint-based,  opportunistic  model  of  operator  scheduling  behavior  was  developed 
using  the  blackboard  architecture  provided  as  part  of  the  Cenenc  ExprtSy  stemT^ 
CGEST)  Display  portions  of  this  component  are  written  in  Symbolics  Common  Lisp. 
Sid  a taskqueue  of  indeterminate" length,  along  with  data  a*>ut 
logical  constraints,  estimated  duration,  resource  requirements,  etc.),  Z solves  tor  a near 
optimal  seauence  and  schedule  based  on  a strategy  of  either  time  minimizing  or  load 
balancing^intended  to  represent  possible  operator  behaviors  The  scheduler  contains 
modular  components  or  knowledge  sources  that  represent  individual  stages  in  the 
scheduling  process  with  an  extended  task-based  decomposition  (a  divide-and-conquer 
technique! used  to  partition  the  overall  scheduling  problem.  Z closel^meracts  wiA  the 
MIDAS  task  loading  model  for  reasoning  about  resource  interactions  between  plaus 
concurrent  tasks.  Refer  to  Annex  B for  a more  detailed  description. 

5.1. 1.1.2  Task  Loading  Model 
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^,h,n?r«?I!ent’  'I?ttCn  in  SymboIics,  Common  Lisp,  is  based  on  current  research  in 
multiple  resource  theory,  scaling,  workload,  and  perception.  Based  on  attributes  of  the 
mission  tasks,  world  state,  operator,  and  crew  station  equipment,  a resource  classification 
ttxonomy  is  used  to  classify  individual  tasks  in  terms  of  their  demands  on  the  Visual 
Auditoiy,  Cognitive,  and  Motor  processing  dimensions.  In  addition,  conflict  matrices  are 
used  to  describe  the  interactions  of  these  resource  demands  across  different  processing 
dimensions  and  tasks.  Refer  to  Annex  C for  a more  detailed  description.  ^ 

5.1. 1.2  Symbolic  Equipment  Models 

These  generalizable  structures  written  in  Symbolics  Common  Lisp,  allow  characteristics  of 

10  represented  in  terms  of  both  physical  and  functional 
USCd  t0  sPfciahze  *e  mission  tasks  prior  to  a simulation.  In  this 

Can,S^Pf)0rt  th£,pnera0on  of  exPlicit  operator  actions  which  are  sensitive 
to  specific  equipment  designs.  These  structures  also  provide  a model-based  means  to 
maintain  and  manipulate  equipment  state  variables  which  drive  the  animation  of  graphical 
cockpit  controls  and  displays.  Refer  to  Annex  D for  a more  detailed  description.  P 

5. 1.1.3  Visual  Editor  and  Simulation  Tool  (VEST) 

^cSn^HaTcintTintiVe  3uD  tOQl  used  t0  create’ control-  ^ observe  from  several  visual 
perspectives,  a 3-D  graphic  representation  of  vehicles  traversing  through  DMA  terrain 

fhTmfccirt  mU  ad°n‘  UsCrS  Can  -elect  by  mouse  a viewing  position  from  anywhere  within 
the  mission  gaming  area,  zoom  in  on  specific  controls  and  displays  for  study,  as  well  as 
include  a representation  of  the  Jack  Anthropometric  model  within  the  crew  station  for 
visualizing  operator  movement  during  a simulation.  The  Cockpit  Design  Editor  (CDE)  a 
subsystem  of  VEST,  contains  interactive  3-D  modeling  utilities  for  graphically  prototyping 
the  crew  station  geometry  instruments,  controls,  and  displays  using  a built-in  libraryof  g 
primitive  cockpit  objects.  Links  can  be  made  to  other  models  or  data  files  for  animation  of 
selected  controls  & displays.  A significant  extension  has  recently  been  made  to  this 
component,  allowing  the  creation  and  animation  of  multi-function  displays  (MFD) 
cajt^g  grapluMl  features,  text  strings,  and  dynamic  fields.  This  component  is  written  in 

mSSS??  SfwV*  1°"“?  modehn§  package  from  Software  Systems,  Inc.  called 
MuItiGen  . Refer  to  Annex  E for  a more  detailed  description, 

5.1. 1-4  Display  Layout  Analysis  (DLA)  tool 

The  Display  Layout  Analysis  (DLA)  tool,  is  an  initial  prototype  of  a tool  intended  to  assist  a 
crcwstation  designer  in  laying  out  the  displays  which  provide  the  pilot  with  windows  to  the 

“ ?rU°n  SOUr,Cej  ch  he/she  needs  to  successfully  operate  the  aircraft.  Display  layout 
assistanceis  guided  by  a set  of  design  metrics  incorporated  into  the  tool,  which  can  be  y 
°^a  e.d  h01*1  *n  an  analyfic,  design-aiding  mode  and  in  an  evaluative  mode.  In  the  analytic 
mode,  human  factors  design  guidelines  form  networks  of  relations  between  information^ 
sources  and  other  information  sources,  between  information  sources  and  controls,  and 
between  information  sources  and  regions  of  the  display  surfaces.  A rule-based  advisor  can 
be  invoked  which  issues  warnings  for  detected  violations  of  display  layout  guidelines 
This  component  is  written  in  C,  using  the  Expert  System  shell  CLIPS  for  the  rule-based 
advisor.  Refer  to  Annex  F for  a more  detailed  description. 

5.1. 1.5  Anthropometric  Model  (Jack) 

n53,,D'  dynamic  anthropometric  model  has  been  developed  through  a grant  to  Dr.  Norman 
Badler  at  the  University  of  Pennsylvania  in  order  to  address  fundamental  human 
anthropometry  and  motion  considerations.  This  model,  called  Jack,  is  written  in  C and 
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runs  on  the  Silicon  Graphics  4D  series,  providing  realistic  and  physically  quantifiable 
human  figure  motion  within  a 3-D  space  environment  Jack  aDowthe ^ to  re) lect 
different  sized  human  figures  or  graphic  mannequins  that  include  tiie  5th  through  95th 
nercentile  male  and  female,  based  on  NASA  astronaut  demographics.  These  mannequins 
cm  then  be  placed  within  a 3-D  object  environment  created  and  stored  using  a number  of 
modeline  packages  Articulation  is  achieved  using  a goal-solving  technique  based  on 
SSng  body  joint  orientations  or  end-effector  (limb)  goals.  Joint  limitations  have  been 
£lkd  to  eliminate  unreasonable  movements.  Kinematic  and  inverse  kinematic  controls 
are  applied  so  that  goals  and  constraints  may  be  used  to  position  and  orient  the  figure,  with 
extemal/intemal  forces  and  torques  applied  to  produce  motion.  A movement  time 
calculation  has  been  incorporated  based  on  Fitts  Law,  using  reach  site  distance  and  target 

width. 

Supporting  graphic  output  in  wireframe,  solid  filled,  or  smooth  shaded  modes,  key  poses 
can  be  stofeffi  interpolated  for  animation,  allowing  envu^nmental  hmrauons  to  be 
detected  as  a function  of  human  size  and  movement  characteristics.  In  addition,  by 
attaching  the  "view"  of  the  environment  to  the  mannequin  s eye.  Jack  displays  a 
PCTSDective  corresponding  to  what  the  mannequin  would  "see  while  moving  indie 
environment,  providing  the  first  step  toward  further  analysis  arid  conclusions  about  object 
occlusion  and  visibility.  Refer  to  Annex  G for  a more  detailed  description. 

5. 1.1.6  Vision  Models 

Refer  to  Annex  H for  a more  detailed  description  of  the  Vision  Models. 

5.1. 1.6.1  Volume  Field  of  View  Model 

This  model  of  binocular  human  visual  representation  in  3-D  space  was  developed  by  Dr. 
Aries  Arditi  at  The  Lighthouse  of  New  York.  It  provides  computer  graphic  methods  for 
delineating  and  testing  hypotheses  about  the  relationship  between  tw<>dimensional  visu 
field  maps8 and  the  three-dimensional  visual  space  they  serve,  under  the  conditions  of . 

1 ) Sng  eye  position;  2)  occlusion  by  structures  that  are  part  of  or  are  mounted  on  the 
observer  such  as  facial  structures,  goggles,  or  headgear,  3)  occlusion  by  environmental 
objects-  4)  normal  and  abnormal  defects  of  the  visual  field  such  as  blind  spots  and  areas  of 
temporarily  reduced  visibility  due  to  local  adaptation  and  photopigment  bleaching,  and 
5)  variables  that  alter  the  focus  of  environmental  objects  on  the  retinas  (accommodation  and 
pupillary  response).  Instantaneous  field  of  view  volumes  based  on  these  factors  are 
visualized  by  projecting  their  intersection  with  the  object  space  in  different  colors.  Th 
model,  written  in  C,  is  fully  integrated  with  the  Jack  anthro^metnc  m^el  which  is  used 
to  determine  the  operator's  head  position  and  point  of  regard  in  the  field  of  view. 

5.1. 1.6.2  Cockpit  Display  Visibility  Model 

This  analytical  model,  written  in  C and  also  fully  integrated  into  the  Jack  environment  was 
devdoned  bv  Drs  Jim  Bergin  and  Jeff  Lubin  at  the  SRI/David  Samoff  Research  Center.  It 
allows  die  designer  to  asses8  the  visibility  of  cockpit  objects  imaged lor > the  ret main 
of  a visual  system  footprint.  This  footprint  represents  the  projection 
of  the  sensory  capabilities  of  the  human  visual  system  when  consider^s  a detector/m 
system.  The  existing  MIDAS  graphical,  anthropometric,  and  vision  modeling  capabilities 
are  used  to  describe  the  physical  characteristics  of  potential  designs  and  define  the 
instanraneousvolume  field  of  view.  Based  on  such  information,  this  component  provides 
methods  to  project  the  retinal  photoreceptor  apertures  onto  the  cockpit  model  Md  support 
"mpSyKed  predictions about  die  legibility  of  characters  and  symbol  Because  the 
human  retina  is  highly  inhomogeneous,  the  retinal  footprint  produced  is  also  hig  y 
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inhomogeneous,  depicting  contours  of  visual  performance  data  which  describe  the 
probability  that  certain  imaged  information  will  or  will  not  be  legible.  Because  factors  such 
as  ambient  illumination  in  the  cockpit,  the  adaptive  state  of  the  operator,  and  the  reflective 
/emissive  properties  of  displays  are  critical  to  consider  in  such  contexts,  this  model 
addresses  each  of  these  aspects. 


5.1.1.7  Aerodynamics  & Guidance  Model  (AGM) 

The  MIDAS  AGM  is  a two-part  Fortran  model,  initially  developed  by  Analytical  Mechanics 
Associates  Inc.,  which  represents  rather  generic  helicopter  guidance  and  dynamics  for 
uncoupled  controls.  Given  the  current  position,  orientation,  and  angular  rates,  the 
guidance  portion  of  the  model  determines  the  control  inputs  required  to  fly  to  the  next 
waypoint  with  its  associated  position,  altitude,  and  airspeed.  The  aerodynamics  portion  of 
the  model  uses  the  computed  controls  to  determine  the  helicopter's  next  position, 
orientation,  etc.,  based  on  the  simulation  tick  interval.  The  AGM's  input  and  output  is 
integrated  with  the  symbolic  pilot  model  and  anthropometric  model  such  that  during  a 
simulation,  the  computed  flight  control  requirements  are  passed  to  the  symbolic  pilot  model 
as  resource  demands,  with  their  actual  start  times  and  duration  determined  by  the  evaluation 
of  such  demands  in  the  context  of  other  pilot  psychomotor  activities.  Flight  control 
movements  are  graphically  depicted  by  attaching  the  Jack  anthropometric  model's  end 
effectors  to  the  appropriate  controls  and  using  inverse  kinematics  to  "pull”  the  appendages 
to  the  computed  control  positions.  Refer  to  Annex  I for  a more  detailed  description. 

5.1. 1.8  Simulation  Executive  and  Communications  Module 


This  component,  written  in  both  C and  Lisp,  uses  TCP/IP  protocol  to  facilitate  inter- 
machine communication  and  dynamic  message  sharing  between  all  MIDAS  components.  A 
data  pool  concept  is  used  during  the  simulation  to  synchronize  and  distribute  in  excess  of 
200  operator,  world,  and  equipment  state  variables  among  the  simulation  processes  and 
objects.  Refer  to  Annex  J for  a more  detailed  description. 


5.1. 1.9  Training  Assessment  Module 


The  training  assessment  module  provides  heuristic  methods  to  the  nwtia 

instructional  techniques,  and  time  necessary  to  train  various  operators  to  "initial 
qualification"  based  on  characteristics  of  the  operator,  task,  and  crew  station  equipment 
pus  prototype  knowledge-based  system  is  implemented  in  ART™  (the  Automated 
Reasoning  Tool)  and  Common  Lisp  on  the  Symbolics.  This  tool  uses  the  instructional 
systems  design  (ISD)  methodology  to  assign  each  task  a set  of  learning  experiences  (such 
as  explanation,  demonstration,  part-task  training,  and  full  task  training)  along  with  a 
medium  for  each  learning  experience  (such  as  textbook/workbook,  slide/tape,  lecture 
videodisc,  and  a wide  range  of  simulation  devices).  For  each  learning  experience  and 
media  assignment,  a time  to  train  is  computed,  based  on  the  task,  operator,  and  equipment 
attributes.  This  module  was  developed  in  Phase  III;  detailed  documentation  is  included  in 
the  Phase  III  Detailed  Design  Document. 


5.1.2  Demonstration  Scenario 


The  Phase  IV  demonstrations  consisted  of  a 15  minute  introductory  briefing  by  the 
program  manager,  followed  by  a little  more  than  two  hours  of  demonstrations  by  the 
development  staff.  The  demonstration  objective  was  to  describe  and  demonstrate 
individual  components  of  MIDAS  as  well  as  to  demonstrate  the  integrated  mission 
simulation  capability.  The  demonstrations  made  use  of  data  for  controls,  displays,  mission 
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profiles,  and  operator  task  descriptions  obtained  from  the  on-going  McDonnell  Douglas 
AH-64  Apache  Longbow  Program. 

_ • • -twhff  mF  the  nrocedures  to  build  and  animate  a set  of  displays/controls  on 

gssssssseassss- 

possible  violations  of  human  factors  principles. 

S to  determine  feasibility  of  reaches.  The  interactive  user  interface  of  SAS,  a 
spreadsheet  containing  anthropometric  data,  was  also  demonstrated. 

The  Vision  Models  were  then  demonstrated  in  the  J^k  enviioi^nt  j^e  ° 

Rn3  The visibility  model  was  demonstrated  using  data  for  the  O and  Q 
conditions  of  ambient  lighting  and  multi-function  display  luminance. 

mmme 

Psychomotor  (VACP)  load  limits,  and  temporal/logical  constraints, 
of  tasks  was  shown. 

networked  together  during  the  integrated  dynamic  simulation  run. 
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The  demonstrations  were  concluded  by  demonstrating  the  fully  integrated  dynamic  mission 
simulation  capability  of  MIDAS.  The  helicopter,  with  fully  piuh^cockSdSs 
of  the  pilot  and  copilot/gunner  (CPG),  was  placed  in  the  gaming  area,  driven  by  the 
aerodynamics  and  guidance  model,  and  viewed  from  three  perspectives,  a world  view 
showing  die  hehcopter  moving  over  the  DMA  terrain,  a view  ofthe  pilot  cockpit  fern  over 
his  shoulder  and  a view  of  the  CPG  cockpit  from  over  his  shoulder. 

The  program  goal  for  Phase  IV  had  been  to  be  able  to  demonstrate  the  simulation  of  a riven 
mission  scenario  in  two  cockpit  environments,  the  Apache  AH-64A  and  the  Apache  ** 
Longbow  and  to  obtain  comparative  data  across  the  two  designs  given  the  same  mission 
goals.  In  the  event,  however.  Phase  IV  demonstrations  included  only  the  Lonebow 
cockpit  configuration.  The  simulated  mission  ran  at  approximately  ten  times  real  time 

de^nSon6  Adhnnf  °nly  * “"ft?  segment  of  the  simulated  mission  scenario  for  the 
' k short.sefm.ent  °J the  scenario  was  shown,  in  which  the  CPG  selected  a 
radio  frequency  by  manipulating  the  multi-function  display  pages. 

5.1.3  Programmatic  Information 

5.1.3.1  Constraints 

Following  the  Phase  IV  off-site  at  Asilomar  Conference  Grounds  July  26-28  1989  the 
group  began  design  and  development  work  for  Phase  IV.  Near  the  end  of  the  Phase  IV 
^VeA  3rPrnTl  Penr°d’,ajuni0r  C.  Programmer.  Dr.  Christian  Neukom,  was  hired  to  become 
the  A I in-house  Jack  expert.  Just  before  the  demonstration  period,  Dr.  Betsy  Constantine 
was  hired  as  Task  Manager  for  the  Sterling  staff.  By  the  end  of  Phase  IV  the  in-house 
staff  consisted  of  3 Lisp  programmers  , 4 graphics  programmers,  a human  factors 
specialist  / junior  Lisp  programmer,  a Task  Manager  with  a Lisp  background,  a system 
administrator  and  a CAD  draftsperson.  ^ U Q’  a system 

a SymboUcS  MaC,vo^  ^ Purchased  to 

lhe  Ljghthouse  of  New  York  and  the  SRVDavid  Samoff  Research  Lab  was 
continued  for  applied  vision  models.  These  components  have  proven  to  be«tremelv 
valuable  models  for  MIDAS,  eliciting  considerable  interest  at  the  demonstrations  y 

The  new  organization  plan  for  A3I,  which  established  two  principal  scientist  positions  and  a 

i”mDlem^H  SThranSfer  P Called  the LIndustry  Liais°n  Section  has  been  partially 

' The  fortunate  to  have  been  able  to  fill  the  Principal  Scientist  for 

P°sm°n  by  hl?nS  Dr-  Kevm  Corker,  a cognitive  scientist  with  a long  history  of 
nvolvementwith  the  A3I  program.  He  brings  exciting  new  concepts  in  the  simulation  and 

in  8 t0the  progT.^T1  aJld  wil1  be  directing  the  research  and  design  efforts 

in  Phase  V.  Dr.  James  Lanmer  is  still  acts  in  the  role  of  Principal  Scientist  for  Perception 

5.1.3.2  Risks 

Selection  of  the  appropriate  level  of  detail  at  which  MIDAS  design  and  analysis  tools  are 
int^d^  to  operate  remains  a difficult  issue.  The  A3I  Program  Office  has  made  it  clear  that 

the^ith  wrf  f°r  the  c.oncePtua]  development  phase  of  crewstation  design  because  of 
g p yoff  for  properly  incorporating  human  engineering  principles  during  this 
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period.  However,  it  is  becoming  increasingly  clear  that  most 

models  and  analysis  methods  currently  or  potentially  incorporated  in  I^AS  requuj  as 
inDuts  task  equipment,  and  environmental  data  which  is  more  appropriate  for  detai 
H£  TOtoSSwit  conflict  between  the  model/analysis  needs  and  the  intended  use  of 
MIDAS  is  still  unresolved.  It  may  be  true  that  even  though  ^AS  ^ntend^  to  be  used 
in  the  early  conceptual  stages  of  design,  its  use  may  require  a degree  of detailed  analyse 
and  specification  that  is  not  usually  performed  at  the  conceptual  design  Mibb.  This  need 
may  require  a change  in  the  design  process  and  could  have  serious  implications  for  the 
Program's  success  in  developing  a prototype  workstation  which  meets  the  needs  of  its 
projected  users. 

5.1.3.3  Summary  of  Results 

There  was  a very  positive  response  to  the  traditional  end-of-phase  demonstrations^  Begun 
in  June  1990,  these  demonstrations  were  attended  by  more  than  200  J^AS  * 

the  US  Army,  other  DoD  components,  several  universities  as  well  as  from  industry, 
particularly  the  major  helicopter  manufacturers. 

There  was  a great  deal  of  interest  in  the  Scheduler  with  its  two  scheduling  strategies,  both 
as  evidence  of  real  progress  toward  cognitive  modeling  and  for  its  potential  stand-alone 
usefulness  There  were  questions  about  whether  the  scheduling  strategiesthat  were 
implemented  actually  represented  human  operator  scheduling  behavior.  This  question  will 
be  addressed  in  Phase  V when  empirical  data  will  be  obtained  in  an  attempt  to  validate  the 
results  obtained  from  the  Scheduler. 

The  Task  Loading  Model  was  of  considerable  interest,  particularly  to  industry  visitors, 
who  must  produce  loading  estimates  for  their  designs.  Again,  the  biggest  wsuejnu 
whether  the  MIDAS  Loading  Model  had  been  validated.  In  Phase  V there  will  be  a i high 
priority  placed  on  obtaining  empirical  data  appropriate  for  use  m validaung  the  Task 
Loading  Model. 

The  Display  Layout  Analysis  tool  was  a big  hit  with  many  visitors,  partly  because  it 
incorporates  a very  effective  visualization  technique  that  immediately  connects  with 
observer  and  partly  because  it  was  quite  clear  just  how  a designer  might  use  such  a tool.  It 
demonstrates  direct  aiding  of  designers  as  they  solve  a specific  design  problem  and 
provides  a clear  means  of  evaluating  a proposed  display  configuration  according  to 
established  human  factors  principles. 

The  Vision  Models  elicited  a great  deal  of  interest  from  most  visitors.  Design  questions 
IbouTSilhy  « easy  to  understand  and  are  key  issues  to  be  addressed  dunng  die  design 
process  The  Vision  Models,  integrated  into  the  Jack  environment,  present  attractive  a 
useful  visualizations  of  the  kind  of  answers  a designer  might  want  from  such  a tool.  These 
models  were  seen  as  useful  interactive  tools  for  a static  analysis  of  visibility. 

On  the  slightly  negative  side,  the  integrated  simulation  ran  so  slowly  dial  it  was  difficult  to 
imaeine  how  a designer  would  interact  with  the  dynamic  analysis  aspect  of  MIDAS.  A 
decision  was  made  St  to  artificially  speed  up  the  demonstration  by  running  the  graphics 
from  files  obtained  earlier,  because  it  was  important  to  have  visitors  un^rstand  that  true 
integration  of  many  of  the  modules  involved  in  the  mission  simulation  had  been  achieved 
The^Scheduler  and  Task  Loading  Model,  however,  while  integrated  with  each  other,  were 
not  integrated  with  the  Symbolic  Operator  Model  during  the  actual  demonstration,  although 
the  required  communication  protocols  had  been  established. 
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A few  important  things  were  missing  from  MIDAS  at  the  time  of  the  demonstrations 
^grated  .analysis  capability,  although  the  Symbolic  Operator 
Model  collected  data  during  the  simulated  mission  scenario  that  could  have  served  as  raw 
data  for  analysis.  Also,  the  intention  had  been  to  demonstrate  a comparison  of  two 
different  cockpit  designs,  the  Apache  Longbow  and  AH-64A,  with  the  same  mission 
scenario.  This  comparison  was  not  available  and  the  simulated  mission  scenario  was  run 
with  only  one  cockpit,  the  Longbow. 

5.2  DETAILED  DESIGN 


Please  refer  to  Annexes  A-J  for  detailed  design  information  for  each  component  of  MIDAS. 
6.0  USER’S  GUIDE 


Please  refer  to  Annexes  A-J  for  User  Guide  information  for  individual  modules. 
7.0  ABBREVIATIONS  AND  ACRONYMS 


A3I 

AGM 

AMA 

APU 

ART 

BBN 

CAD 

CBT 

CDE 

CFT 

CLIPS 

CPG 

CPS 

CPT 

CSCI 

DTED 

DMA 

DOF 

EES 

HF/CAE 

I/O 

ISD 

MFD 

MIDAS 

NFS 

NRC  CoHF 

OFT 

SCDD 

SGI 

TCP/IP 

USGS 

VEST 

WST 


Army-NASA  Aircrew/Aircraft  Integration 
Aerodynamics/Guidance  Model 
Analytical  Mechanics  Associates,  Inc. 

Auxiliary  Power  Unit 

Automated  Reasoning  Tool 

Bolt,  Beranek  and  Newman  Laboratories,  Inc. 

Computer-Aided  Design 

Computer-Based  Training 

Cockpit  Design  Editor 

Cockpit  Familiarization  Trainer 

C Language  Information  Processing  System 

Copilot/Gunner 

Computer  Program  System 

Cockpit  Procedures  Trainer 

Computer  Software  Configuration  Item 

Digital  Terrain  Elevation  Data 

Defense  Mapping  Agency 

Degrees-of- Freedom 

Expert-EASE  Systems,  Inc. 

Human-Factors  Computer-Aided  Engineering 
Input/Output 

Instructional  Systems  Development 
Multi-Function  Display 

Man-machine  Integration  Design  & Analysis  System 
Network  File  Software 

National  Research  Council  Committee  on  Human  Factors 
Operational  Flight  Trainer 
Software  Component  Description  Document 
Silicon  Graphics  Inc. 

Transmission  Control  Protocol/Intemet  Protocol 
United  Stated  Geological  Survey 
Visual  Editor  and  Simulation  Tool 
Weapon  System  Trainer 


8.0  NOTES 
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8.1  LIMITATIONS 

One  critical  limitation  of  Phase  IV  MIDAS  is  the  lack  of  any  kind  of  user  interface  for 

™c ! L^gbow' 5i!  de“  g?  Smildtave  to  encode  *“ 

St  equipment  models  and  equipment-dependent  activities  in  Common  Lisp. 

Another  limitation  in  the  usefulness  of  MIDAS  to  potential  desi^r/“^sp^S^e 
of  the  cockoit  modeling  and  simulation  graphics  on  the  MultiGen  packag  . 
TbeUcense  fees  for  MultiGen™  are  high  enough  to  stand  as  a considerable  barrier  to  e 
dissemination  of  MIDAS. 

8.2  FUTURE  DIRECTIONS 

A t the  transition  between  Phase  IV  and  Phase  V,  MIDAS  is  conceived  as  having  two 

fmproves  ^present  iterative,  man-in-the-loop  design  process,  an^ysisandevduation  of 
Slation  results  will  be  emphasized  and  more  complete  integration  of  all  components  of 
stress^.  For  the  simulation,  software  wiU  conform  to  a highly 
distributed  object-oriented  architecture,  sometimes  called  an  agents  ot  actors  architecture. 
SSectTe  ^ will  enhance  the  ability  of  MIDAS  to  function  as  a framework  for 
incorporating  a variety  of  models  developed  elsewhere,  for  as  long  as  a model  cor^orms 
the  message8passing  protocols  of  agents  with  which  it  must  communicate,  it  can  be 
!nS3  without  disturbing  the  functioning  of  the  remainder  of  the  system. 

At  the  end  of  Phase  V,  MIDAS  is  envisioned  as  running  on  two  machines  networked 
meetherone  Symbolics  machine  and  one  SGI  IRIS  machine.  Fori die  interact!’ ve  tools  a t 
common  graphics  environment,  running  on  the  SGI  IRIS  machine,  will  be  chosem  Th 
graphics  environment  must  not  have  serious ^cumb^ces,  like *«^®nsefees or 

the  Symbolics  and  IRIS  machines. 


Durinc  Phase  V,  a high  priority  will  be  given  to  establishing  collaborations  with 

M LDA^f  modelsl'p  ^ c uTarly^  he" Task  SchJIgM^Sd  the  Schedule.  Attention  will  be 
given  to  validating  other  models  as  they  are  developed. 

9.0  HISTORICAL  INFORMATION 


9.1  PHASE  I DEVELOPMENT 
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9.1.1  Requirements  and  Design  Approach 

9.1. 1.1  Summary  Level 


The  initial  phase  of  development  of  the  prototype  HF/CAE  workstation  found  the  Program 
in  senous  danger  of  extinction  due  to  insufficient  funding  caused  by  regular  and  sizable 
cuts  from  guidance  funding  levels.  It  was  believed  that  a visually-compelling,  proof-of- 
concept  demonstration  was  required  to  communicate  the  essence  of  the  Program  to 
individuals  unfamiliar  with  the  particulars  of  the  science  involved.  Maximum  visual  utility 
was  demanded  of  every  expenditure  and  development.  y 

A baseline  simulation  capability  was  required  that  demonstrated  mission  modelling,  human 
performance  metrics  and  helicopter-pilot  interactions  within  a controllable,  time-stepped 
environment  that  provided  multiple  graphic  "views"  into  the  underlying  model(s)  ThT 
simulation  needed  to  be  incrementally  extensible,  hence  only  a framework  for  more 
elaborate  modelling  was  required  for  this  phase,  given  the  time  constraints  imposed, 
several  areas  of  development  emphasis  were  identified: 

9.1. 1.2  Mission  Modelling 


The  overall  A3I  Program  model  architecture  calls  for  a mission  model  driving  the  closed 
loop  pilot-vehicle  system.  The  model  would  be  developed  with  a dynamic,  interactive  task 
analysis  framework  for  systematically  describing  tasks  involved  in  certain  classes  of 
advanced  helicopter  operations.  The  framework  will  provide  a feasibility  demonstration  of 
the  methodology,  including  all  critical  mission  and  flight  management  functions  within  a 
pre-spccified  sample  scenario. 

Typically,  scout-attack  helicopter  missions  are  largely  opportunistic  or  discretionary  in 
nature.  Consequently,  the  mission  model  generated  by  the  mission  decomposition 

m.ust  allow  ***  component  to  be  represented,  either  by  providing  conditional 
branching  based  on  some  pilot  model  parameter,  or  stochastic  event  triggering. 

Bolt,  Beranek  and  Newman  (BBN)  had  already  started  development  (under  a NASA 
contract  initiated  pnor  to  the  PDR)  of  a "Mission  Decomposition  Methodology"  that  would 
provide  essentially  the  entire  simulation  structure  for  Phase  I.  Refer  to  the  respective 
software  component  description  document  for  the  Phase  I Mission  Editor  for  details. 

In  order  to  drive  and  manipulate  the  specific  mission  model  generated  by  the  Mission 
Decomposition  Methodology,  a simulation  executive  was  required  that  allowed  greater  user 
control  of  the  simulation  than  conventional  executive  programs.  Future  integration  of  a 
vast  number  and  variety  of  models  within  this  executive  structure  was  projected  as  well 
hence  flexibility  as  well  as  functionality  was  required.  Refer  to  the  software  component 
description  document  for  the  Phase  I Modeller  for  details. 

Communications  software  was  required  to  link  the  Symbolics  3670  running  the  mission 
model  with  the  Silicon  Graphics  IRIS  2500T  displaying  dynamic,  3-D  graphic  views 
dnvcn  by  the  mission.  The  link  was  Ethernet  under  TCP/IP  protocol.  Refer  to  the 
software  component  description  document  for  Phase  I Communications  for  details. 

9.1. 1.3  Graphics 
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3-D  color  dynamic  mission  representation  displays  were  required  to  provide  intuitive 
understanding” of  simulation  progress.  Further,  these  so-called  views  became  extremely 
valuable  as  drugging  aids  for  programmers 

the  software  component  description  document  for  the  Phase  I Graphic  Views  for  details. 

In  addition  to  view  graphics,  a state  display  editor  was  required  to  allow  destam  to  select 
appropriate  model  variables  for  run-time  observation,  and  determine  bow ^“^^eretli 
values  were  to  be  displayed.  This  tool  allowai  designers  to  individually  »lect  which 
simulation  variables  were  of  interest  for  monitoring,  and  the  nature  of  their  ^splay  (i.e. 
dial,  bar,  graph,  etc.).  Refer  to  the  software  component  description  document  for  the 
Phase  I Icon  Editor  for  details. 


9.1. 1.4  Human  Performance  Modelling 

The  first  phase  needed  to  demonstrate  the  capability  to  model,  structure  and  rndyze  the 
human  component  of  complex  and  interactive  pilot-helicopter  systemsby  elucidating  die 
effects  of  human  performance  limitations  on  mission  effectiveness.  The  mission  had  to  be 
responsive  to  changes  in  pilot  performance,  and  conversely,  the  pilot  s loadings  should  be 
reflected  in  task  loadings  imposed  by  execution  of  the  prescribed  mission. 

It  was  decided  that  emphasis  should  be  placed  on  developing  some  meaningful 
demonstration  of  training  effects  as  provided  by  profiles  of 

representations.  Refer  to  the  software  component  description  document  for  Phase  1 
Training  Implications  for  details. 


9.1. 1.5  Demonstration  Scenario 

The  demonstration  scenario  consisted  of  the  capability  to  perform  multiple  consecutive 
simulations.  Variables  included: 

1 ) A novice  and  experienced  pilot  profile  that  may  be  menu-selected 
prior  to  a simulation  run  to  illustrate  training  effects. 

2)  Convoy  return  of  missile  fire  at  any  point  in  the  mission  subject 
to  the  discretion  of  the  designer. 


It  was  possible  to  have  extensive  run-time  control  over  the  running  of  models  from  me 
items.  It  was  also  possible  to  examine  data  and  information  both  after  the  run,  and  du  g 
model  freeze  state.  Menu  selection  of  these  capabilities  required  no  programming 
experience  to  start,  operate  and  evaluate  the  simulation. 


9.1.2  Hardware  Environment 

Figure  7 below  indicates  the  Phase  I hardware  configuration.  These  components  are 
described  in  further  detail  in  the  subsections  which  follow. 


Page  33 


Figure  7.  Phase  I Hardware  Configuration 


9.1.2.1  Symbolics  Lisp  Machines 

Model  3670-1433  Color  Workstation  consisting  of: 

8MB  Main  Memory 

Ethernet  Controller  and  Transceiver 

335  MB  Fixed  Disk 

Monochrome  Console 

Keyboard  Sc  Mouse 

SYS36  20  MB  System  Software 

Documentation 

Model  CG70-FB02  High  Resolution,  24-bits/Pixel  Color  Frame  Buffer 
Model  CCOP-OIL  19"  Color  RGB  Monitor 
Model  OP36-FPA1  Floating  Point  Accelerator 
Model  CGSW-PKG  Software  Package  consisting  of: 

SCGR-DYNA  Dynamic  Animation  System 
SCGR-PAINT  Paint  System 
SCGR-GEOM  Geometry  System 
SCGR-RENDER  Rendering  System 
Symbolics  # S LAN -FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 

9.1. 1.2  Silicon  Graphics  Computers 

W-2500A  Workstation  consisting  of: 

1/4"  Tape  Drive 

HU-T04  Turbo  Option  W/4MB  RAM 
H3-FPA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z Clipping  Assy, 

C-WTCP IP/TCP  Software 
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P-DBX  Di»l/Button  Box 
R-UNIF  Fortran  Compiler 

9.1.2.3  Other  Processors 
None. 

9.1.2.4  Networking  Hardware 

TCL  Incorporated  Model  2010EC  Ethernet  Transceivers  (tap-type) 

9.1.2.5  Peripherals 

Okidata  Model  2410  Dot  Matrix  Printer 


9.1.3  Software  Environment 

v the  oranhic  design  and  analysis  tools  were  built  using  C,  Unix,  and  the 
Replimre'3-D  Selling  package  for 

displays  is  shown  in  Figure  8 below. 


Figure  8.  Phase  I Software  Modules 


9.1.4  Programmatic  Information 
9.1.4.1  Constraints 

The  demonstrations  had  to  be  conducted  prior  to  July,  1986  hence  there  was  little  time  to 
conducted  on  25  October  1985,  where  general  objectives  were  established. 
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Staffing  initially  included  1 EE  (software  integration  task  manager),  and  1 systems 
programmer.  A LISP  programmer  was  added  in  October,  1985,  followed  by  a senior 
graphics  specialist  is  Apnl,  1986  and  an  entry-level  programmer  in  May,  1986.  Training 
and  familianty  was  required  for  any  programmer  working  with  the  Symbolic  computer 
(often  quoted  to  be  a 6-month  learning  curve).  y computer 

Due  to  the  staffing  limitations,  subcontracts  were  required  to  perform  some  of  the 
necessary  work.  In  particular,  3-D  graphic  view  work  was  initially  subcontracted  to  a 
small  business  marketing  a 3-D  CAD  modelling  package.  Deliver^  of  the  vSk  wa? 
ultimately  several  months  late,  and  incomplete,  causing  in-house  programming  staff  to  be 
severely  pressed  to  integrate  and  debug  other  software  prior  to  the  established 

deadl‘ne.  Future  contractual  endeavors  cannot  involve  organizations  that  are 
criticd  path  hlmsnreSPOnS1Ve’  °r  th3t  “*  devel0ping  that  may  be  considered 

Jteh^m?wned  a Symbolics  3670  LISP  computer  with  24-bit  color  system  at  the  time 
SmTP^hircPrOCUrement  S^0rtiy  afterward  10  acquire  a Silicon  Graphics  IRIS 
condidM  S' VST'  “ ‘n  bUI  w“  "O'  ^ worid"8 


9.1.4. 2 Risks 

TTie  majority  of  Phase  I applications  developments  were  under  contract  or  subcontract 
This  condition  poses  some  risk  of  failure  caused  by  potential  incompatibilities  at  the  time  of 
integration,  competing  priorities  inherent  within  respective  organizations,  lack  of  control 

SommoS  an^°gress’ and  otScr  Problems  that  cannot  be  treated  or  corrected  by 
the  Program  Office.  While  appropriate  for  this  initial  phase  (due  to  constraints)  each  X 

Since  the  Phase  I architecture  was  minimal,  there  were  few  technical  risks  (outside  of 
»dCa2incCS)  *hat  might  prohibit  success.  The  major  source  of  uncertainty  involved 
" thf  Symbolics  and  IRIS  through  TCP/IP  Ethernet.  Unfortunately  late  delivery 

graphics  software  forced  a suboptimum  (almost  frenzied)  approach  to  communications^ 
debuppng,  since  in-house  staff  did  not  have  the  necessary  familiarity  with  TCP/IP  protocol 
details.  Consultants  were  used  to  assist  with  debugging  and  optimizing  the  link. 


9.1.4.3  Summary  of  Results 


ftehminary  demonstrations  (of  the  mission  model)  were  held  in  February  1986  followed 
by  the  first  of  several  formal  Phase  I demonstrations  starting  in  late  June  1986  The 
demonstrations  consisted  of  three  computer  displays  (two  in  color): 


1 ) Three-dimensional  (3D),  color,  dynamic,  mission  representations 
composed  of  world,  pilot  and  plan  views  of  the  simulated  mission. 

2)  Mission  model  set-up,  control  and  data  display. 

3)  Color  iconic  "state  displays"  providing  continuous  display  of  mission  model 
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The  simulation  executive  (on  the  Symbolics  3670)  cotmolling  tte  nririn  moM  ^jded 
appropriate  data  via  EtherNet  TCP/IP  to  3D  graphic  views  resident  on  the  DUS  250OT  to 
drive  each  dynamic  simulation  object.  Mission  model  programming  began  mNovemberof 
1985,  state  display  work  began  about  the  same  time,  while  graphic  dismay  w . 

late  January  1986.  Over  50  man-months  of  combined  programming  effort  wa^edicated 
to  this  Phase  generating  nearly  5000  lines  of  Fortran  code  (3000  m-house,  2000  contact), 
1200  lines  of  C (in-house),  and  7800  lines  of  Lisp  (3800  in-house,  4000  contract)  m this 

eight  month  period. 

Numerous  design  and  implementation  compromises  were  made  in  the firet  phase  < of 
development  due  to  the  severe  time  constraints  (start  11/1,  complete  by  mid  June),  and 
minimal  staffing  available.  Future  phases  must  address  more  long-term  strategic 

approaches. 

9.2  PHASE  II  DEVELOPMENT 

9.2.1  Requirements  and  Design  Approach 

9.2.1. 1 Summary  Level 

Phase  D was  intended  to  devise  more  long-term  approaches  to  the^deveiopment  of  the 
workstation.  Another  primary  purpose  was  to  assemble  a ream  of  individuals  ^thin 
house  and  outside)  appropriate  for  this  activity.  This  team  would  be  c°m^s^°f 
researchers  and  implemented,  since  the  Program  s approach  is  to  employ  contempomy 
techniques  to  integrate  the  best  models  of  human  behavior/performance,  vehicle/systems 
and  environment  available. 

Phase  I had  succeeded  in  demonstrating  that  the  concept  of  a prototype 
aiding  early  helicopter  cockpit  design  with  regard  to  human  perfom^ce  limitau mw 
viable.  The  implications  of  attempting  to  develop  such  a system  were  also  more  c e y 
understood  after  Phase  I.  The  purpose  of  Phase  II  was  to: 

1)  Develop  modelling  and  simulation  tools 

2)  Develop  graphics  tools 

3)  Integrate  6 DOF  helicopter  dynamics 

4)  Develop  a more  representative  mission 

5)  Design  and  implement  a more  modular  architecture 

6)  Gain  additional  insight  into  modelling  and  the  design  process 

7)  Build  an  appropriate  in-house  implementation  team 

8)  Establish  working  relationships  with  various  centers-of-excellence 

Ai  Ac  completion  of  Phase  I it  was  evident  that  mote  long-tent,  stntte»es  u>  dewlopment 
would  have  to  be  adopted  if  the  Program  were  to  ultimately  succeed.  It  was  also 
recognized  that  a more  active  effort  was  necessary  in  the  area  of  integration  of  research 
resufts  if  less  mature  fields  such  as  human-computer  interaction  and  predictive  human 
modelling  were  to  gain  any  acceptance. 

New  software  development  requirements  in  Phase  II  centered  around  modelling 
envtio^rptiotmSdels.  veLle/systettts  models, 

aiding,  and  user  interfaces  Work  Breakdown  Structure  (WBS)  elements.  The  specific 
applications  chosen  are  described  in  the  subsections  that  follow. 
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9.2.1.2  Modelling  Environment 
9.2. 1.2.1  Mission  Editor 

Development  of  the  Mission  Editor  continued  in  Phase  II  under  subcontract  with  BBN  with 
domain  expanse  and  integration  supplied  by  in-house  staff.  BBN  designed  a graphic 
editing  interface  utilizing  the  mouse  and  pop-up  menu  templates  to  relieve  the  user  of  Lisp 
code  editing.  A manual  decision  interface  was  also  implemented  to  override  the  previous 
selection  by  aspect  method  of  simulated  pilot  decision-making.  The  Mission/Task  Editor 
k an  application  initially  developed  BBN  that  serves  as  the  human  performance  modelling 
framework  whereby  more  elaborate  computational  models  of  human  behavior  and  6 
performance  may  be  installed  or  "activated"  in  the  workstation  contingent  on  the  type  and 
level  of  analysis  required  to  answer  a particular  question.  For  example,  the  framework 
provides  a default  toplevel  directed  acyclic  graph  form  of  human  task  modelling  that  can  be 
used  to  evaluate  task  sequencing  and  resultant  human  resource  loadings  (visual,  auditory 
cognitive  and  psychomotor)  based  on  empirical  data.  However,  it  is  possible  to  integrate 
detailed  predictive  computational  models  of  human  performance  such  as  dynamic  6 

anthropometric  models  that  are  able  to  compute  rates,  durations,  reach,  comfort  factors  and 
other  parameters.  These  detailed  models  can  be  utilized  as  an  alternative  to  the  more 
subjective  toplevel  empirical  models  that  the  system  provides  by  default.  The  framework 
also  provides  interfaces  to  simulation  models  of  the  vehicle/systems  and  environment  at 
various  levels  of  abstraction.  Most  importantly,  the  framewoiic  provides  contingent  task 
behavior  subject  to  vehicle  and  environmental  state  variables.  Refer  to  Appendix  1 of  the 
Phase  II  System  Architecture  Description  Document  (SADD)  for  details. 

9.2.1. 2.2  Modeller 

The  Modeller  serves  as  the  Phase  II  simulation  executive.  It  provides  all  modes  of 
interaction  with  the  simulation,  including  build/edit,  test/verify,  experiment  frame,  run 
analysis  and  document/report.  The  ultimate  goal  of  the  Modeller  is  to  provide  the  complete 
^\r°nr^5nt  for  construction,  integration,  testing,  simulation  analysis  and  reporting  of 
results.  The  most  developed  mode  of  the  Modeller  is  the  run  mode,  whereby  the  user 
controls  the  execution  of  the  simulation  models.  Model  selection,  data  specification  and 
run-time  display  (state-displays  and  views)  configuration  is  provided  through  the 

details™111  framC  mode  of  the  Modeller-  Refer  to  Appendix  2 of  the  Phase  II  SADD  for 
9*2. 1.2.3  Visual  Modeller 

^^fl^MOC!eller1iS  a,proto!ype  visual  Programming  language  for  dynamic  systems 
modelling  developed  under  subcontract  by  Expert-Ease  Systems  Inc,  of  Belmont,  CA.  It 

f°*s‘he  user  sel^t  components  such  as  dividers,  summers,  integrators,  sources  and 
sinks  from  an  extensible  library  of  components  and  assemble  them  on  a graphic 
worksheet  to  form  working  models.  Connections  between  component  input  and  output 
ports  are  make  graphically,  as  is  specification  of  initial  conditions  and  parameters  Models 
are  subsequently  run  as  interpreted  code  (versus  compiled)  subject  to  boundary  conditions 
(start  time,  end  time,  step  size)  supplied  by  the  user.  The  application  was  initially 

A eed  as. a stand'a,lone  t0Ql  for  evaluati°n,  hence  it  has  yet  to  be  integrated  with  other 
MIDAS  workstation  elements.  It  is  anticipated  that  some  form  of  visual  programming 
language  will  be  developed  for  the  Modeller  build/edit  mode  that  utilizes  the  same  type  of 
interface  as  the  Visual  Modeller.  This  tool  would  become  the  primary  means  of  developing 
and  mte^aung  vehicle/systems  and  world  models  for  the  simulation.  Refer  to  Appendix  3 
of  the  Phase  II  SADD  for  details. 
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9.2. 1.2.4  State  Display  Editor 

The  State  Display  Editor  is  an  enhanced  version  of  the  Phase  Hcon  &Utor,  which  isbased 
on  the  Graphics  Editor  from  Steamer,  an  application  developed  by  BBN  for  theNavy 
Personnel  Research  and  Development  Center  (NPRDC)  in  San  Diego,  CA.  The  S 
Display  Editor  is  used  to  conveniently  build  displays  composed  of  graphs,  bars,  disds, 
sliders  text  etc  for  indicating  the  state  of  some  selected  dynamic  variable  dimng  a 
simulation.  ’The  user  interface  has  been  substantially  reworked  to  ™nnc  the  Appk 
Macintosh  desktop  metaphor.  It  also  takes  advantage  of  improvements  to  the  Symtohcs 
operating  system  (Genera  7.1)  such  as  infinitely  scrollable  windows  in  both  vertical  and 
horizontal  dimensions.  Other  features  added  to  the  original  Icon  Editor  include 
"Macintosh-like"  text  handling,  keystroke  commands  (to  supplement  pull-down  menus) 

issssxid ss fT  ssTASSJar"' 

is  required  to  use  this  tool.  Refer  to  Appendix  4 of  the  Phase  II  bADD  tor  oetaus. 

9.2.1.3  Pilot  Models 

9.2.1. 3.1  Anthropometric  Model 

The  A3I  Anthropometric  Model  was  developed  under  a NASA/Ames  grant  to  Dr  Norman 
Ftadler  at  the  University  of  Pennsylvania’s  Department  of  Computer  and  Information 
Science  The  POSIT/HIRES  model  provides  human  body  dimensions,  reach  and  position 
based  on  CAR  (Crewstation  Assessment  of  Reach,  Boeing  Coip.)  database  link  sizes 
driven  bv  task/goal  specifications  (HIRES).  Unlike  other  systems  that  statistically  utilize 
anthropometric^ databases,  POSIT  allows  the  user  to  specify  each  link ^Sow^^c 
pctahlkh  motion  constraints  for  any  link  or  joint  in  the  model.  Since  the  anthropometric 
SSeHs  ingoing  Search  effort  at  the  University  of  Pennsylvania,  a workmg  version 
nf  POSIT  was  not  received  by  the  Program  until  1 month  pnor  to  the  end  of  Phase  II. 
ConfeaLT  ?he  Suties  of  the  system  were  demonstrated  off-line  as  a stand-alone 
wsreKoyugh^ ^graphic  databases  generated  by  other  MIDAS  workstation  tooh  were 
iSlaTely  made  compatible  with  POSIT/HIRES,  as  well  and  the  Missior^Task  Editor  s 
numut  Future  Phases  of  the  Program  will  see  integration  of  the  next  generation  o 
pnsiT/HTRES  fJACK/GOALTENDER),  which  is  expected  to  include  dynamics  and  field- 
SSg  Refer  .0  Appendix  5 Sf  the  Phase  D SADD  for  de.a, Is. 

9. 2. 1.3.2  Loading  Model 

The  loading  model  currently  used  to  measure  human  performance  is  based  on  data 
SeurSlSTrte  Amy  Research  Institute  at  Ft.  Rucker,  AL,  that  was  stdrseqnendy 
commit  to  computer  hardware  within  a modelling  framework  developed  by  BBN.  The 
model  generally  states  that  task  performance  requires  human  resources  from  visua  , 

(VACP)  dimensions.  This  approach  is  lcx>sely based 

on  Chris  Wickens  Multiple  Resource  Theory  and  subsequent  r on  a 

Aldrich  & McCracken  at  Anacapa  Sciences  Incorporated. 

Qnrvev  of  active  helicopter  pilots  who  were  asked  to  estimate  the  VA  E> 

survey  of  active  nencopier  p specific  guidelines  and  criteria  for  each  level  in 

to  support  alteration  of  baseline  VACP  values  as  a function  of  these  variables. 
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9.2.1. 4 Vehicle/Systems  Models 

9.2.1.4.1  Dynamics  and  Guidance  Models 

Hl!ctynl0f  hclicoPl"  dynamics  and  guidance  models  used  in  Phase  I were  discrete  point- 
mass.  These  models  did  not  provide  the  necessary  data  in  translational  and  rotational 

t0  andyZC  V?l]lde  Mentation  (pitch,  roll,  yaw)  as  a function  of  control 
movements.  The  dynamics  model  used  in  Phase  n had  been  used  at  Ames  previously  in 
nrotion  simulation  studies  on  VAX  VMS  hosts,  thus  the  majority  of  the  effort  required  to 
use  tins  model  was  a port  to  the  Symbolics  computer.  This,  however,  was  nota simple 

dlffercnces  and  *c  Program's  need  to  "package"  models  in  a mtSiular 
fashion  that  simplifies  integration  and  interaction  with  other  code,  most  notably  Liso.  The 
guidance  routines  were  also  based  on  existing  code  used  in  motion  simulator  facilities, 
mthough  extensive  enhancements  were  required  and  performed  by  Anil  Phatak  and  Huan 
Tran  of  Analytical  Mechanics  Associates  (AMA),  Inc.  These  included  outer-loop  speed, 
horizontal  and  vertical  path  control  feedback  to  provide  path  following  that  was  more  ’ 
representative  of  real  pilot  strategies.  An  important  lesson  learned  in  Phase  II  was  that  it  is 
not  advisable  to  attempt  to  integrate  models  that  are  still  undergoing  development  into  a 
system  that  itself  is  evolving,  particularly  when  there  are  language  compatibility 

^1,-a!1T  ^g-Fortran  and  Lisp).  Although  this  was  necessary  in  Phase  II  due  to  time 
constraints,  future  Phases  should  avoid  this  through  better  planning. 

9.2.1.4.2  Cockpit  Display  Editor 

^d“0r  (<?E)  is  f new  aPPlication  developed  for  the  construction  and 
ediung  of  cockpit  displays  and  controls.  Its  interface  is  based  on  the  MultiGen™  visual 
database  ediung  program  developed  by  Software  Systems  of  San  Jose,  CA.  Software 
Systems  and  Sterling  Software  entered  a joint  development  agreement  which  allowed 
Sterling  source  code  rights  to  the  MultiGen™  software  in  exchange  for  developing 
enhancements  to  MuluGen™  (described  in  the  Graphic  Views  SCDD).  The  CDEis  3-D 
color,  dynamic,  and  has  a user  interface  modelled  after  the  Apple  Macintosh.  Displays  can 
be  conveniently  linked  to  model  parameters,  or  animated  from  stored  datafiles.  The  CDE 
contains  the  database  ofall  positional  and  geometric  attributes  of  displays  and  controls  in 

C^k?U'  database  wlU  eventually  be  utilized  by  human  behavior  and 
performance  models  such  as  signal  detection  and  visibility  to  analyze  optimal 
instruments/control  placement.  3 v 


9.2. 1.5  World  Models 

9.2.1.5.1  World  Models 

World  models  have  not  changed  significantly  over  Phase  I,  with  the  exception  of  replacing 
flat  terrain  and  gaussian  hills  with  Defense  Mapping  Agency  (DMA)  terrain  data. 

considerable  effort  was  devoted  to  separating  the  simulation  of  world 
bjects  from  pilot  and  vehicle/systems  simulation  models  in  anticipation  of  migration  to 
distributed  or  parallel  processing  hardware.  ^ 

9.2.1.5.2  Views 

?H^^^CfrePi,reSenLaSn,S  °n  W™d  ?bjects  have  c^geri  appreciably  from  Phase  I with  the 
addinon  of  enhanced  MultiGen™  software.  As  mentioned  in  world  models,  flat  terrain  and 
gaussian  hills  have  been  replaced  with  (DMA)  terrain  data,  and  objects/vehicles  have  real- 
world  dimensions  since  they  were  obtained  form  a simulator  out-the-window  visual  system 
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database  (Singer-Link  DIG).  The  DMA  data  can  be  read  directly  off  a tape  and  transformed 
into  a 3-D,  colored  image  by  the  graphic  modelling  system.  The  £ 

been  enhanced  as  well,  since  MultiGen™  features  provide  interactive,  Macintosh-like 
editing  of  objects  (it  is  an  object-oriented  editor)  and  subsequent  viewing  with  3 
translational  and  rotational  degrees  of  freedcm^  ModeJ-^yen  of 

objects  is  also  provided  as  a feature  that  was  added  to  the  basic  MultiGen  software. 

9.2. 1.6  Analysis  and  Decision  Aiding 

9.2. 1.6.1  Training  Resource  Requirements  Prediction 

The  impact  of  training  was  demonstrated  in  Phase  I by  providing  2 types  of  pilot  models 
(skilled  and  novice)  and  comparing  the  performance  results  in  a simulated  mission  for  ea 
case  The  two  models  were  based  on  the  assumption  that  on  the  average,  a less 
pilot  requires  longer  and  perceives  a higher  (VACP)  load  in  task  performance  than  a skilled 
Dilot  Phase  II  endeavored  to  demonstrate  that  it  would  be  possible  to  perform  an  analysis 
Sf  a simSed  ^ssion’s  tasks  and  extrapolate  the  d-aining  notice  requuements  necessary 
to  train  an  individual  to  perform  such  a mission.  The  approach  drew  heavily  from 
SSSSS System  Development  (ISD)  methodologies.  Hence  the  focus  was  upor .post- 
simulation analysis  and  training  requirements  estimation,  rather  that  attemptmgtoshow  the 
effects  of  skill  level  variations  on  mission  performance.  Both  are  considered  important  for 
the  MIDAS  workstation. 


9.2. 1.7  Demonstration  Scenario 


The  Phase  II  Scenario  involved  walking  demonstration  attendees  through  a simulated 
mission  and  cockpit  build,  anthropometry  analysis,  and  task  Joading/tmiehn^ns^ctiotL 
Emphasis  was  placed  on  "running"  as  many  components  together  in  an  integrated  fashion 
for  a slight  derivative  of  the  Ambush  Scenario  first  started  during  Phase  I. 


9.2.2  Hardware  Environment 

Figure  9 below  indicates  the  Phase  II  hardware  configuration.  These  components  are 
described  in  further  detail  in  the  subsections  which  follow. 
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Figure  9.  Phase  II  Hardware  Configuration 


9.2.2. 1 Symbolics  Lisp  Machines 
Model  3675  Color  Workstation  consisting  of: 

Monochrome  Console 
Keyboard  & Mouse 
45  MB  1/2**  Cartridge  Tape  Drive 
Ethernet  Controller  and  Transceiver 
12  MB  RAM 

Enhance  Performance  Option 
474  MB  Fujitsu  Eagle  Disk 
515  MB  CDC  Disk 

Model  CG70-FB02  High  Resolution,  24-btts/Pixel  Color  Frame  Buffer 

Tektronix  19”  Color  RGB  Monitor 

Model  OP36-FPA1  Floating  Point  Accelerator 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 

Symbolics  # STCP-1  TCP/IP  Software 
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Model  3640  Color  Workstation  consisting  of: 

Monochrome  Console 

Keyboard  & Mouse 

Ethernet  Controller  and  Transceiver 

6 MB  RAM 

2-140  MB  Disks 

CAD  Buffer 

Tektronix  19"  Color  RGB  Monitor 
Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 

Model  3620  Monochrome  Workstation  consisting  of: 


Monochrome  Console 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
4MB  RAM 
368  MB  Disk 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 

9.2.2.2  Silicon  Graphics  Computers 
W-2500A  Workstation  consisting  of: 

45  MB  1/2"  Cartridge  Tape  Drive 
Ethernet  Controller  and  Transceiver 
HU-T04  Turbo  Option  W/4MB  RAM 
H3-FFA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z CUpping  Assy 
C-WTCP  IP/TCP  Software 
P-DBX  Dial/Button  Box 
R-UNIF  Fortran  Compiler 

W-3120  Workstation  consisting  of: 

Ethernet  Controller  and  Transceiver 
H3-FPA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z Clipping  Assy 
C-WTCP  IP/TCP  Software 
P-DBX  Dial/Button  Box 

9.2.2.3  Other  Processors 
Digital  Equipment  Corp.  MicroVax  II 


9.2.2.4  Networking  Hardware 

TCL  Incorporated  Model  2010EC  Ethernet  Transceivers  (tap-type) 


9.2.2.S  Peripherals 

Okidata  Model  2410  Dot  Matrix  Printer 
Apple  LaserWriter  Plus  Laser  Printer 
GraphOn  Graphics  Terminal 
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9.2.3  Software  Environment 

th^"rer«  wj!!I?  gr^?lic  desiJ.n  “d  anaJysis  tools  were  built  using  C,  Unix  BSD  4 3 and 
^ Wtndow  Manager/Graphics  Library  II.  The  MultiGen™  modelling  package  was 
used  as  the  underpinnings  for  the  CDE  and  Views  components.  Mo«S^u?d 
models  were  built  using  Symbolics  Common  Lisp  under  Genera  6 2 with  the  Flavors 
extension,  and  the  S-Packages  were  often  used  for  for  Displays.  Communication 
software  was  written  to  enable  inter-machine  message  passing  and  simulation 

Rgure°0  below.  ^ dlStnbution  of  ^ Phase  11  models, tools,  and  displays  is  shown  in 


TCP/IP  ETHERNET 


S.G.IRIS  EDITOR7  DISPLAY  Is.G  IRIS  I VISUAL  DATABASE  I 

3120  2500T  EDIT0R  SYMBOLICS  D/IPS 

POSIT  ANIMATION  EXEC.  ^ *«• 


MIISSION 

EDITOR 


MODELS 


Cockpit  Build 
Anthropometry 
Analysts 


Iris  Color  Display 
OFFLINE 


Iris  Color  Display 
3-D  "VIEWS” 


Models 

Editors 

Inspectors 


Sym..  B&W  Display  Sym..  Color  Display 
SIM.  EXEC  DISPLAYS 


Figure  10.  Phase  II  Software  Components  & Displays 

9.2.4  Programmatic  Information 
9.2.4. 1 Constraints 

fhe,uhaSe  I!°[fsi[e  Plannin.g  meeting  at  Saint  Francis  Retreat,  a schedule  was 
de  eloped  for  the  work  that  had  been  identified  during  the  planning  meeting  The  schedule 

TSTi  th,at  ^ en/  ?f  PhaSe  11  would  re<*uire  a Period  of  apSrS  f ve^ 

fntin°U'^^  tbe  efldA.°^P.hase  demonstrations  would  as  a consequence  likely  miss  affecting  the 
following  year's  funding  cycle,  the  plan  was  accepted.  y g thc 

Phase  II  development  did  not  start  until  in  late  December,  1986  due  to  the  need  to  nrenare 
extensive  budget  proposals  while  at  the  same  time  developing  technical  plans  At  thaf time 
here  were  6 in-house  programming  staff  members  composed  of  3 iSSSSmS 
(2  junior,  1 intermediate),  2 graphics  programmers  (1  senior,  1 junior)  a systems 
administrator  and  a task  manager  to  perform  and  coordinate  the  implementation  of  the 

Sareh^OsTli! 1 apPhcatl0ns.in  cooperation  with  2 subcontractors  (BBN  and  AMA) 

In  March,  1987,  the  senior  graphics  programmer  left  the  Program  to  head  another  project 
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followed  by  the  systems  administrator  shortly  thereafter.  These  individuals  wwe  replaced, 
as  well  as  the  addition  of  4 new  staff  member  and  4 subcontracts  by  July,  1987. 


A decision  was  make  in  July  to  complete  Phase  II  by  early  October  and  begin  Phase  ll  in  an 
effort  to  complete  Phase  III  in  time  to  influence  the  following  year  s funding.  Although  the 
staff  required  to  perform  work  planned  for  Phase  II  were  not  available  until  very  near  the 
end  ofthe  phase,  and  the  length  of  the  phase  was  reduced  by  2 months,  sufficient  work 
was  completed  to  demonstrate  the  essential  concepts  planned  for  Phase  11. 

Equipment  was  a severe  limitation  until  the  delivery  of  two  additional  Symbolics  computers 
and  an  IRIS  workstation  by  June.  Additional  mass  storage,  dynamic  memory  and 
hardware  upgrades  were  ordered  for  existing  equipment  With  the  exception  of  the 
Symbolics  upgrade  to  a 3675,  most  of  the  hardware  was  received  and  installed  in  time  to 

improve  the  productivity  of  staff. 


9.2.4. 2 Risks 

By  the  end  of  Phase  II  there  were  6 subcontracts  and  1 NASA  grant  under  the  management 
of  in-house  implementation  staff.  This  represents  a sizable  nsk  that  the  cont^utions  of 
these  subcontractors  be  appropriate,  cost-effective  and  carefully  monitored.  The  limited 
funds  of  the  Program  could  not  tolerate  any  waste  of  manpower  or  funds. 

The  decision  to  attempt  to  integrate  discrete-event  Lisp-type  and  continuous  Fortran-type 
models  in  Phase  n,  without  the  benefit  of  a specialist  in  simulation  modelling,  could  have 
proven  to  be  a disaster.  Although  successful,  the  process  was  inefficient  and  frustrating. 
Future  attempts  at  such  model  integration  must  be  made  with  the  aid  of  an  expert  in 
simulation  modelling. 

Hardware  technological  risks  were  again  minimized  by  the  use  of  standard,  Prpv®" 
computers  and  software.  Communications  was  TCP/IP  protocol,  and  special  hardware  had 
been  purchased  to  monitor  network  data  packets  in  the  event  of  any  problems. 

The  composition  of  the  in-house  implementation  staff  is  currently  engineers,  programmers, 
mathematicians,  computer  scientists,  physicists  and  a helicopter  pilot  There  are  no 
cognitive  psychologists  or  human  factors  specialists.  This  is  a senous  void  for  a team 
developing  a prototype  workstation  for  human  factors  analysis.  Nonetheless,  this  void  has 
been  temporarily  filled  by  subcontracting  for  this  expertise. 


9.2.4.3  Summary  of  Results 

The  first  set  of  demonstrations  was  conducted  for  NASA  and  local  personnel  on  October 
22  1987  followed  by  numerous  scheduled  and  non-scheduled  demonstrations  over  die 
course  ofthe  next  6 weeks.  The  feedback  was  without  exception  ^.tivedthoughit 
became  clear,  after  several  comments,  that  in  the  next  phase  of  development lt  wo“'° . 
necessary  to  address  some  concrete  design  issues  rather  that  remaining  general  and  abstract 
with  regard  to  what  the  MIDAS  workstation  could  accomplish. 

Late  in  Phase  II  the  Program  Office  obtained  a Chief  Scientist  to  direct  ongoing  research 
activities  that  are  slated  for  eventual  integration  in  the  workstation.  Tnis  addition 
Sendously  improves  the  Program's  ability  to  evaluate  and  respond  to  new  research 
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results  that  may  be  relevant  to  the  design  of  complex  man-machine  systems.  Similarly,  two 
consultant  subcontractors  involved  with  the  Program  have  extensive  experience  in  the  field 
of  human  behavior  and  performance  modelling. 

As  Phase  II  was  aimed  at  more  long-term,  strategic  approaches,  there  will  be  a substantial 
amount  of  code  reuse  as  well  as  coding  momentum  carried  into  Phase  ID.  This  momentum 
is  further  facilitated  by  reasonable  success  in  assembling  a team  of  individuals  with 
appropriate  skills  and  interests  for  this  Program.  These  interests  include  human 
behavior/performance  modelling,  training,  graphical  modelling  languages,  integrative 
frameworks,  decision  aiding  and  user  interfaces. 

9.3  PHASE  III  DEVELOPMENT 

9.3.1  Requirements  and  Design  Approach 

9.3.1. 1 Summary  Level 

Responses  to  the  Phase  II  demonstrations,  as  well  as  discussion  at  the  Phase  III  off-site 
reinforced  the  need  to  continue  the  development  of  the  core  set  of  A3I  models  and  tools. 
However,  emphasis  would  have  to  be  placed  on  explicitly  addressing  how  such  models 
and  tools  would  be  sensitive  to  cockpit  design  change.  Furthermore,  the  enhancement  of 
the  present  applications  and  the  start  of  new  components  must  be  anchored  to  a "real 
world"  application  for  effective  demonstration  positions.  The  Program  office  decided  to 
use  a detailed,  vertical  slice"  of  the  conceptual  development  process  as  a method  to 
illustrate  the  intended  use  of  the  workstation.  The  AH-64A  mission  and  cockpit  was  the 
focus  of  the  phase,  with  empirical  flight  test  results  from  an  AH-1  Cobra  Communication 
j j as-,a  S0UI]ce  specific  task  data.  These  objectives  required  a degree  of  integration 
and  detail  previously  impossible,  and  drove  the  following  development  requirements  for 
the  individual  MIDAS  components: 

9.3.1.2  Symbolic  Modelling  CSCI 

Consisting  of  a mission  editor,  task  decomposition  aids,  as  well  as  state  displays  for  task 
analysis,  the  focus  for  the  Symbolic  Modelling  CSCI  during  this  phase  was  on  the  design 
and  coding  of  a generalizable  framework  for  symbolically  representing  the  functions  of 
cockpit  equipment  used  to  accomplish  mission  tasks.  This  framework  allows  various 
cockpit  alternatives  to  be  evaluated  without  completely  re-editing  the  mission 
decomposition,  since  the  design  maintains  a distinction  between  the  physical  structure  (or 
state  operators)  of  the  equipment,  and  the  functional  requirements  (or  inferred  goals) 
required  by  the  task.  Previous  A3I  symbolic  models  of  the  mission  and  pilot  tasks  failed  to 
explicitly  depict  the  relationship  of  the  equipment  design  and  the  primitive  task  actions 
loading  values,  or  timelines.  Results  of  the  "vertical  slice"  example  (report  phase  tine) 
were  used  to  demonstrate  this  process  and  successfully  compared  to  actual  data  from  a 
similar  AH-1  flight  test  conducted  by  the  Aeroflightdynamics  Directorate.  Task  timeline, 
resource  use,  and  loading  displays  were  similar  in  concept  to  those  used  during  previous 
pnases. 

9.3.1.3  Graphic  Views  CSCI 

Phase  III  development  requirements  for  the  Views  CSCI  are  best  described  as 
enhancements  to  the  well-received  capabilities  existing  in  Phase  II.  The  internal  resolution 
of  the  geometric  modelling  package  was  reduced  from  3/8  inch  to  1/256  inch,  permitting 
sharper  and  more  detailed  rendering  of  the  DMA  terrain  and  moving  models  made  possible. 
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wafiwmd  toX'new  IRIS^D  a^mSedas  dicX  by  a new  version  of  the  underlying 
MultiGen™  software. 

9.3. 1.4  Cockpit  Design  Editor  (CDE)  CSCI 

U ictino  rnp  software  was  ported  to  the  IRIS  4D  and  improved  with  the  addition  of  pop- 
up user8 windows,  an  hierarchical  data  base  of  instrument  mfo^^ 
for  human  factors  analysis,  and  improved  mouse  operations.  The  CDE  was  successru  y 
used  to  build  a detailed  3-D  model  of  the  complete  AH-64A  pilot  cockpit  and 
instrumentation,  providing  both  a feasible  application^ 

brought  about  by  a new  version  of  the  underlying  MultiGen  software. 

9.3.1.5  Anthropometric  Model  or  JACK  CSCI 

parts,  Umb  joint  constraint  settings  adjustable  eye  vie^  ^ 

S^OTEaXSaS 

mnrlp^  and  a new  sliding  window  user  interface.  Jack  was  used  during 
&£££££ visual  occlusion  ics.  using  5ih  and 

95th  percentile  males/females  in  the  Apache  Cockpit 
Far  more  rigorous  vision  models  were  kicked-offlhis 

3S5SS5ESKSSbs±.“- 

not  expected  until  Phase  IV . 

9.3. 1.6  Aerodynamics  & Guidance  Module  (AGM)  CSCI 

TV,,.  &r.M  rsn  was  ported  from  the  Symbolics  computer  to  the  IRIS  4D  and  almost 
^?^nslated Tom  Fortran™  C to  (ake  advantage  of  the  additional  compute  power. 

iirilirliiPMi- 

\Ar*A* inner  rcn'c  was  attemoted  but  made  difficult  due  to  limitations  oi  tne  aum  i 
accurately  pom^ingtome  pllng  activities  requimd  in  the  demonsnanon  scenano. 
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9.3.1. 7  Communications  CSCI 


The  IRIS  to  Symbolics  Communications  software  developed  during  Phase  n was  used 
essentially  as  is  for  Phase  III.  Some  modification  was  necessary  due  to  the  receipt  of 
two  new  IRIS  s during  the  phase.  Because  most  of  the  application  developments 
addressed  during  this  period  were  "stand-alone"  oriented,  the  major  use  of  this  CSCI  was 
to  transmit  posihon,  orientation,  rate,  and  engine  characteristics  from  the  AGM  to  the 
Views  and  CDE  CSCI's  for  animation. 

9.3. 1.8  Training  Assessment  CSCI 

Training  assessment  was  greatly  augmented  and  refined  during  Phase  m.  During  Phase  II 
an  instructional  system  was  assigned  (by  table  look-up)  to  train  each  task  by  matching  task ' 
characteristics  with  attributes  of  instructional  systems  within  the  task’s  learning  category. 
The  Phase  III  approach  used  ART  (the  Automated  Reasoning  Tool)  and  Common  Lisp  on 
the  Symbolics  to  deve  op  a prototype  knowledge-based  system.  This  processes  uses  the 
instructional  systems  design  (ISD)  process  to  assign  each  task  a set  of  learning  experiences 
(such  as  explanation,  demonstration,  part-task  training,  and  full  task  training)  alongwith  a 
medium  for  each  learning  experience  (such  as  textbook/woikbook,  interactive  slide/tape 
JSXI?  TlSU^  m?s,videodisc/CBT,  CFT,  CPT,  OFT,  WST  without  and  with  motion, 
ana  the  actual  system).  For  each  learning  experience/media  assignment,  a time  to  train  is 
computed,  based  on  the  task,  operator,  and  equipment  characteristics.  The  Phase  in 
training  approach  was  heavily  based  on  previous  work  performed  by  the  Logicon 
Coiporation  under  contract  to  the  Air  Force.  Training  assessment  is  accomplished  on  an 

individual  task  basis,  with  no  attempt  made  to  address  the  grouping  of  tasks  into  lessons  or 
courses. 


9.3.1.9  Scheduler 


Work  was  also  begun  this  phase  on  a dynamic  reactive  scheduling  component  to  address 
the  sequencing  and  scheduling  a pilot  may  perform  as  a means  to  control  his  task 
performance  and  timeliness.  While  not  committed  to  code  during  Phase  m,  significant 
headway  into  the  specific  objectives  and  approaches  for  this  state-of-the  ait  component 
£fre  fh/,Ved  Becausei!  *as  not  completed  during  the  phase,  formal  documentation  for 

the  scheduler  is  not  provided  as  part  of  this  Phase  III  SDDD. 

9.3.1.10  Demonstration  Scenario 


The  Phase  III  demonstrations  consisted  of  a 30  minute  introductory  briefing  by  the 
program  director,  followed  by  1.5  hours  of  application  demonstrations  by  the  staff.  The 
vertical  slice  into  the  development  process  was  emphasized  as  the  attendee  was 
essentially  walked  through"  the  conceptual  development  process  for  a potential 
communication  switch  change  on  the  AH-64A.  The  demonstration  objective  was  to 
describe  the  potential  interactions  and  conflicts  arising  from  moving  the  radio  select  button 
from  the  ICS  panel  on  the  front  bulkhead  to  the  cyclic  (as  was  done  in  the  AH-1  flight  test). 
The  capability  for  A3I  to  support  three  phases  of  the  design  process  was  stressed: 
specification,  static  analysis,  and  dynamic  analysis. 


Beginning  with  the  Views  CSCI,  the  projected  DMA  gaming  area  was  portrayed  as  the 
mission  environment  and  the  inherent  capabilities  of  visualization  emphasized.  Next,  the 
mission  editing  component  of  the  Symbolic  Modelling  CSCI  was  introduced,  along  with 
u s facilities  to  input  the  scenario  for  an  unmasking  maneuver  combined  with  several  radio 
calls.  The  Aerodynamics  & Guidance  CSCI  was  then  demonstrated  in  a stand-alone 
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fashion  traversing  over  simulatedflat  terrain  and  viewed  in  several  perspectives. 

Following  the  AGM,  the  CDE  was  demonstrated.  Because  of  the  matunty  and  visual 
nature  of  this  component,  a fair  amount  of  detail  was  provided — beginning  with  the 
procedures  to  build  an  individual  gauge,  attaching  it  to  a control  paneU  m^ung  u ptaang 
it  in  a cockpit,  building  a vehicle  structure  around  the  cockpit,  and  finally,  placuig  the 
S,pSheUcopter  & the  world  prior  to  the  simulation.  As  the  last  demonstfauon  of  the 
MIDAS  specification  capabilities,  the  Symbolic  Modelling  CSCI  was 
time  it  was  used  to  portray  a further  decomposed  mission  with  the 
operator  activities  fully  described  as  a result  of  the  symbolic  equipment  models  for  the 
alternative  communication  switch  configurations. 

Jack  was  then  used  to  perform  some  basic  reach  sequences  in  the  AH-64A  cockpit,  using 
the  maximum  ranges  of  the  human  model  data.  The  fact  that  the  operator  not  n^h 
the  panel-mounted  communications  panel  when  restricted  to  nxmng Y 

(simulating  a locked  inertial  reel)  was  demonstrated.  Additionally,  the  potentially 

dangerous  glare  shield  occlusion  of  the  tailwheel  lock  and  master  arming  switchwas 
shown  for  "taller"  pilots  by  attaching  Jack’s  camera  to  the  mannequin  s eye  during  an 
animation  sequence. 

The  Training  Assessment  CSCI  demonstration  was  then  conducted  for  twosend-mlio 
message  tasks  with  the  alternative  radio  switch  configurations.  The  input,  output,  and 
p^«sing  characteristics  of  this  component  was  described,  as  the  attendees  were  shown 
how  a knowledge-based  system  operates. 

The  newly  initiated  applied  vision  models  were  then  introduced ■ through i a briefing.  An 
Amiga-based  prototype  of  the  New  York  Association  for  the  Blind  s volume  field  of  view 
model  was  used  to  render  the  binocular  retinal  maps,  facial  occlusions,  and  physiological 
blind  spots  for  a series  of  mouse-selected  fixation  points. 

The  demonstrations  were  then  concluded  with  the  dynamic  analysis  capabilities  of  MIDAS. 
The  fully  populated  cockpit  was  placed  in  the  gaming  area,  driven  by  the  aerodynarmes  and 
guidance  model,  and  viewed  from  several  perspectives.  A summary  output  screen  from  a 
"simulation  run"  was  then  shown  on  the  Symbolic  Modelling  color  monitor.  This  screen 
showed  the  loading  and  task  timelines  for  the  two  potential  designs  and  made  use  of 
different  colors  to  describe  physical  resource  conflicts  between  tbc  scenario  sfljnng  tasks 
and  the  radio  selection  actions.  Approaches  for  hypermedia-like  access  toBoff  & 

Lincoln’s  Human  Engineering  Data  Compendium  (once  it  comes  out  on  CD-ROM)  was 
Then  shown  toough  l simulated  key-word  search.  The  intent  was  to  describe  how  analysts 
would  be  able  to  get  extremely  valuable  context-sensitive  infoimauon  for  areas  such  as 
"performance  under  vibration”  or  "effects  of  the  use  of  gloves  to  make  cockpit  design  a d 
mission  decisions. 

9.3.2  Hardware  Environment 

The  hardware  architecture  in  place  at  the  end  of  Phase  III  is  depicted  in 

These  components,  together  with  their  resident  software  and  peripherals  are  described  in 

further  detail  in  the  subsections  which  follow. 
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Figure  11.  Phase  III  Hardware  Configuration 


9.3.2. 1 Symbolics  Lisp  Machines 

Model  3675  Color  Workstation  (Barracuda)  consisting  of: 

Monochrome  Console  with  OCLi  filler 

Keyboard  & Mouse 

45  MB  1/4M  Cartridge  Tape  Drive 

Ethernet  Controller  ami  Transceiver 

22.5  MB  RAM 

Enhanced  Performance  Option 

338  MB  Fujitsu  Eagle  Disk 

550  MB  CDC  Disk 

Model  CG70-FB02  High  Resolution,  24-bits/Pixel  Color  Frame  Buffer 

Tektronix  19"  Color  RGB  Monitor 

Model  OP36-FPA1  Floating  Point  Accelerator 

Symbolics  # S LAN -FORT  Fortran  77  Compiler 

Symbolics  « STCP-1  TCP/IP  Software 

S-Group  (S -Paint,  S-Geometry,  S-Render,  S-Dynamics,  and  color  6.0  V405.13) 
Genera  7.2 


Model  3640  Color  Workstation  (Puffer)  consisting  of: 

Monochrome  Console  with  OCLI  Filter 
Keyboard  & Mouse 
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45  MB  1/4"  Cartridge  Tape  Drive 
Ethernet  Controller  and  Transceiver 


11.25  MB  RAM 
2-140  MB  Disks 
CAD  Buffer 


Tektronix  19H  Color  RGB  Monitor 
Symbolics  # SLAN-FORT  Fortran  77  Compiler 

Symbolics  # STCP-1  TCP/IP  Software  w*nV40S13> 

S-Grouo  (S-Paint.  S -Geometry.  S-Render,  S-Dynam.cs.  and  color  6.0  V405.13) 


Genera  7.2 


Model  3640  Monochrome  Workstation  (Squid)  consisting  of. 


Monochrome  Console  with  OCU  Filler 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
133  MB  RAM 
2-140  MB  Disks 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 

Genera  7.2  . _ 

Automated  Reasoning  Tool  (ART)  Version  3.2 

Model  3620  Monochrome  Workstation  (Sea  Slug)  consisting  of. 


Monochrome  Console  with  OCU  Filter 

Keyboard  & Mouse 

Ethernet  Controller  and  Transceiver 

18  MB  RAM 

190  MB  ST506  Disk 

Symbolics  # SLAN-FORT  Fortran  77  Compiler 
Symbolics  # STCP-1  TCP/IP  Software 
Genera  7.2 


9.3.2.2  Silicon  Graphics  Computers 
W-2500A  Workstation  (Orca)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  & Mouse 
45  MB  1/4"  Cartridge  Tape  Drive 
Ethernet  Controller  and  Transceiver 
12  MB  RAM 

2 474  MB  Fujitsu  10.5”  Disk  Drives 
HU-T04  Turbo  Option  W/4MB  RAM 
H3-FPA  Floating  Point  Accelerator 
H-DM4A  1024x1024x4  Display  Memory 
H-ZC2  Z Clipping  Assy 
C-WTCP  IP/TCP  Software 
P-DBX  Dial/Button  Box 
Unix  System  V with  BSD  4.2 
NFS 

C Compiler 

IRIS  Graphics  Library  U and  Window  Manager 

W-3120  Workstation  (Manta)  consisting  of: 

19”  High  Resolution  Monitor 
Keyboard  & Mouse 
Ethernet  Controller  and  Transceiver 
8 MB  RAM 

72  MB  Winchester  Disk  Drive 
H3-FPA  Floating  Point  Accelerator 
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H-DM4A  1024x1024x4  Display  Memory 

H-ZC2  Z Clipping  Assy 

C-WTCP  IP/TCP  Software 

P-DBX  Dial/Button  Box 

Unix  System  V with  BSD  4.2 

NFS 

C Compiler 

IRIS  Graphics  Library  II  and  Window  Manager 

W-4D120GTX  PowerSeries  Workstation  (Coral)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  Sc  Mouse 
Ethernet  Controller  and  Transceiver 
16  MB  RAM 

380  MB  ESDI  Winchester  Disk  Drive 

Double  Buffered  1280x1024x4  Display  Memory 

Double  Buffered  Alpha 

24  bit  Z buffer 

C-WTCP  IP/TCP  Software 

P-DBX  Dial/Buuon  Box 

IRIX  System  V release  4D1-3.1D 

NFS 

C Compiler 
C++ Translator 
Fortran  77  Compiler 

IRIS  Graphics  Library  II  and  4Sight  Windowing  System 

W-4D20G  Personal  IRIS  Workstation  (Urchin)  consisting  of: 

19"  High  Resolution  Monitor 
Keyboard  Sc  Mouse 
Ethernet  Controller  and  Transceiver 
8 MB  RAM 

170  MB  SCSI  Winchester  Disk  Drive 
1280x1024x4  Display  Memory 
Double  Buffered  Alpha 
C-WTCP  IP/TCP  Software 
IRIX  System  V release  4D1-3.1D 
NFS 

C Compiler 

IRIS  Graphics  Library  II,  4Sight  Windowing  System,  and  Environment  Manager 


9.3.2. 3 Networking  Hardware 

CableTron  MT-800  Ethemet/IEEE  802.3  Transceiver 

9.3.2. 4 Peripherals 


Okidata  Model  2410  Dot  Matrix  Printer 
Apple  LaserWriter  Plus  Laser  Printer 

Seiko  Instruments  D-Scan  CH5312  Color  Printer  & Multiplexor 
GraphOn  GQ-250  ASCII  Terminal  & Keyboard  (2) 
Hewlett-Packard  HP  700/22  ASCII  Terminal  & Keyboard  (3) 

9.3.3  Software  Environment 


^aphic  desj£n  ^lysis  tools  were  built  using  C,  Unix  BSD  4.3,  and 
the  IRIS  Window  Manager/Graphics  Library  II.  The  MultiGen™  modelling  package  was 
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used  as  the  underpinnings  for  the  CDE  and  Views  components,  as  well  a_s  a visualization 
medium  for  theAerodynamics  and  Guidance  CSCI.  Most  Lisp  tools  ^d  m^ds  were  bui 
using  Symbolics  Common  Lisp,  with  the  Flavors  extension,  ? f h 

Training  Analysis  Module  uses  the  Automated  Reasoning  Tool  (ART)  as  a sheHforiu 
inference  requirements.  The  S-Packages  were  available  for  use  in  displays,  although  not 
heavily  relied  upon  during  this  phase.  Communication  software  written  in  previous 
phases  was  used  to  enable  inter-machine  message  passing  and  sumption  s yiKhtom zanon. 
The  distribution  of  the  Phase  ID  models,  tools,  and  displays  is  shown  in  Figure  12  below. 


Figure  12.  Distribution  of  Phase  III  Software  Components  and  Displays 

within  the  A3I  Lab 


Absent  from  the  figure  above  is  the  Communications  CSCI.  This  component  is  actually 
distributed  among  all  of  the  various  Symbolics  and  Silicon  Graphics  machmes  as  dictated 
by  the  integration  requirements.  Currently,  the  capability  exists  to  share  the  following 
variables  among  the  graphic  and  symbolic  computers  during  a simulation. 


For  each  Helicopter  (Ownship  and  Wing):  *Ownship  only 


Helicopter  position  (x,  y,  z) 

Helicopter  orientation  (yaw,  pitch,  roll) 

Altitude  (AGL),  Airspeed,  Groundspeed 
Velocity  in  each  axis 
Engine  torque 

•Cyclic  position,  pedal  position,  collecuve  posiuon 
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For  each  Truck  or  Ground  Vehicle  (up  to  6): 

Truck  position  (x.y^z) 

Truck  orientation  (yaw) 


For  each  Missile  on  board  a Helicopter  or  Ground  Vehicle  (up  to  9): 

Missile  position  (x,  y,  z) 

Missile  orientation  (yaw  & pitch) 


£idJl£?tUni.eaCh  abo^f  ha?  ^ "flaSs" which  can  bet  set  to  communicate  when 
an  event  such  as  a hit  or  an  explosion  occurs. 


9.3.4  Programmatic  Information 


9.3.4.1  Constraints 


Following  the  Phase  HI  off-site  at  Asilomar  Conference  Grounds  in  December  1987  the 

^ !iase  JP 111  earnest.  The  demonstrations  were  initially  set  for  June  of  1988 
but  then  slipped  to  November  1988  for  a variety  of  reasons.  ' 


«,widing  ^°r  fhase  essentially  dictated  a no  growth  policy  for  the  in-house  staff,  alone 
with  minimal  new  subcontracts/grants  or  equipment  purchases.  In  general,  the  computine 
equipment  available  dunng  this  phase  was  well-matched  to  the  development  requirements8 
a rather  distinct  change  from  previous  phases.  In  addition,  shortly  before  the 
demonstrations,  the  A3I  lab  was  completely  remodelled,  greatly  improving  the  staffs 
working  conditions  through  additional  table  space,  improved  lighting,  and  a sharp 
appearance.  Approximately  2 weeks  of  development  time  was  lost  during  this  period 
although  all  the  computing  equipment  managed  to  handle  the  movement  well  The 
additional  space  enabled  us  to  take  receipt  of  a new  SGI  4D70G  IRIS  and  have  it  up  and 
running  quickly.  Subsequent  upgrades  to  this  configuration  allowed  A3I  to  be  one  of  the 
first  users  equipped  with  the  new  PowerSeries  4D120GTX,  containing  parallel  graphics 
Processors  Although  this  capability  was  not  really  exploited  during6 Jie  phase, 
valuable  experience  with  the  new  architecture  was  gained.  F 


In  contrast  to  the  above,  the  impacts  to  staffing  and  outside  efforts  due  to  the  funding  level 
were  not  so  benign.  Overall,  staffing  levels  were  inadequate  to  meet  all  the  goals  set  forth 
by  the  Program  office.  Although  a new  deputy  director  for  A3I  (government  employee) 
was  added  during  August  1988  and  his  background  in  simulation/training  devices  was  very 
helpful  with  the  Training  Assessment  CSCI,  the  crunch  was  felt  by  the  Sterling  Software 
staff.  Throughout  the  majority  of  the  phase,  the  in-house  staff  consisted  of  3 Lisp 

senior,  1 j“ni°r),  3 graphics  programmers  (2  senior,  1 intermediate),  and  a 
system  administrator  who  doubled  as  an  applications  analyst  for  the  Jack  CSCI.  Problems 
with  identifymg  and  hiring  a suitable  Task  Manger  after  Mr  Lakowske's  departure  in  March 
1988  severely  complicated  any  serious  integration  efforts.  The  fact  that  the  A3I  Program 
has  essentially  been  without  a permanent  and  competent  first  line  supervisor  for  over  a year 
has  definitely  delayed  progress  in  general.  It  has  also  made  coordination  and 
communications  between  the  Program’s  diverse  efforts  nearly  impossible. 


To  mitigate  the  staffing  deficiencies,  Dr  Yan  Yufik  (Yufik  and  Associates)  received  a 
subcontract  dunng  this  phase  for  assistance  in  cognitive  modelling  and  training. 
However,  his  ideas  for  an  analytical  complexity  assessment  were  not  universally  well 
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received  and  his  support  was  not  renewed  after  November  1988.  Similarly,  E*  Anil  Phatak 
tom  AMA  aided  the  in-house  staff  with  minor  AGM  improvements,  but  was  also 
unfunded  during  FY89.  An  active  subcontract  with  BBN  was  also  maintained  during  this 

phase  but  not  funded. 

Desnite  the  relatively  weak  year  in  funding,  new  outside  efforts  wereinitiat^withthe  New 
?Sk  Association  tor  the  Bhnd  and  the  David  Samoff  Research  Lab  for  apphed  vision 
mrviels  Soearheaded  by  Dr  James  Larimer,  the  chief  scientist  for  A I,  these  new 
Smponen^Sfto ! be  extremely  valuable  models  for  MIDAS.  However,  to  the 

demonstrations  (June  1988),  Dr  Larimer  was  promoted  to  Branch  chief,  and  while  still 
within  ASHFRD,  his  oversight  and  guidance  for  these  efforts  was  dimimshed. 

Furthermore,  his  more  critical  role  of  overall  scientific  guidance  for  A3I  was  lost  A 
suitable  replacement  has  not  been  found. 

Dr  Larimer's  departure,  combined  with  the  fact  that  virtually  no  one  on  the  development 
staff  was  formally  trained  in  human  factors,  complicated  cur  ability  to  develop  and 
convincbgl^emonstrate  the  use  of  MIDAS  in  a "real  world"  cockpit  design  application. 
The  present  organizational  structure  has  required  each  programmer/analyst  to  be  not  only 
software  developer,  but  a part-time  scientist  and  domain  expert  as  well. 

Following  the  documentation  requirements  at  the  end  of  the  phase,  two  senior  Sr^P’l^L 
programmers  left  the  project  to^ursue  positions  outside  of  Ames.  However,  ttas  penod 
miked  a turning  point.  Crippled  by  the  loss  of  in-house  personnel,  recruiting  activities 
began  immediately.  By  June  1989,  two  outstanding  software  engineers  were  hired.  A 
new  CAD  draftsperson,  a system  administrator,  a human  factors  analyst,  and  a senior  Lisp 
programmer  were  also  hired  during  the  summer,  rounding  out  an  A3I  development  staff 
which  was  now  back  to  full  strength.  Although  a Task  Manager  had  not  been  hired,  we 
had  several  promising  leads  and  were  well-poised  to  enter  Phase  IV. 

On  a related  topic,  a program  office  reorganization  plan  was  approved  by  the  US  Army 

reorganization  expanded  A3I’s  principal  scientist  positions  from  one  to  two,  and .established 
a new  technology  transfer  group  called  the  Industry  Liaison  Section  which  would  be 
headed  bv  a government  project  manager.  A3I’s  core  research  and  development 
responsibilities  would  be  split  between  the  two  scientists,  with  one  steering  f , 

S S other  perceptual  issues.  This  division  is  a sp«:ific  recomme^tion  of  the 
National  Research  Council  study  and  report  entitled  Human 

Computer-Aided  Engineering.  The  dual  scientist  positions  would  hopefully  allow  better 
monitoring  and  guidance  for  our  extramural  grants  and  contracts,  as  well  as  more  time 
devoted  to  and  specific  direction  for  the  in-house  software  development  staff.  The  ILS 
section  would  respond  to  the  increasing  number  of  requests  to^sseminate  in-house 
technology  bv  intensifying  efforts  to  transfer  the  appropriate  A3I  lab  products  to  other 
government  agencies  and  industry.  Unfortunately,  the  filling  of  this  government  post  was 
Phase  IV,  m the  benefits  of  the  new  organize., on  were  no.  realrzed. 

9.3.4.2  Risks 

The  maioritv  of  the  risks  faced  by  A3I  during  Phase  m were  management-oriented  and  not 
technical  in  nature.  They  stemmed  from  unclear  and  conflicting  development  direction 
comSd  wTthe  aforementioned  staffing  situation.  A program  with  die  ambitions  and 
scope  of  this  magnitude  cannot  tolerate  a continuance  of  these  problems. 
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One  quasi-technical  risk  did  exist  and  should  be  mentioned  however.  It  centers  on  the  level 
of  detail  appropriate  for  Ae  MIDAS  design  and  analysis  objectives.  The  Program  Office 
has  made  it  clear  that  MIDAS  is  intended  for  the  conceptual  development  phaseof 
crewstation  design  because  of  the  high  "payoff"  for  properly  incorporating  human 
engineering  principles  during  this  period.  However,  most  of  the  known  human 
performance  models  and  analysis  methods  require  as  inputs  task,  equipment,  and 
environmenta1  data  which  is  more  appropriate  for  detailed  design.  This  appwent  conflict 
between  the  model/analysis  needs  and  the  intended  use  of  MIDAS  was  (and  still  is) 
unresolved.  Its  resolution  will  have  serious  implications  for  the  Program’s  success  in 
developing  a prototype  workstation  which  meets  the  needs  of  its  projected  users. 

9.3.4.3  Summary  of  Results 

had^0ndr  K^nst 10  toe  traditional  end-of-phase  demonstrations. 
Begun  in  November  1988,  these  demonstrations  were  attended  by  approximately  170 
people  from  NASA,  the  US  Army,  other  DoD  components,  as  well  as  several  universities. 

e were  then  asked  by  the  Aeroflightdynamics  Director  to  extend  invitations  to  industrial 
sources,  particularly  the  major  helicopter  manufacturers.  The  detailed  demonstrations 
which  resulted  were  actually  conducted  more  as  joint  working  groups  and  continued 
penodicallythrough  Apnl  1989.  This  activity  precipitated  a significant  amount  of  effort 
which  can  best  be  described  as  technology  transfer.  Lockheed  Missiles  and  Space 
Company  used  our  CDE  package  and  vehicle  dynamics  interface  to  demonstrate  a proposed 

Undenvater  Vehicle  concept.  Boeing  Commercial  Aircraft  CompanyPsp£t 
three  days  at  our  facilities,  understanding  the  tools,  walking  through  code,  and  taking  both 
software  and  documentation  back  to  Seattle  to  set  up  a MIDAS-like  design  workstation  at 
their  company.  Finally,  the  Marine  Advanced  Amphibious  Attack  Vehicle  (AAA  V) 
program  office  became  very  interested  in  the  MIDAS  capabilities,  and  we  completed  some 
h P^ototyP‘;ngdesign  work  for  their  review.  Similar  activities  with  the  Fiber-Optic 
Evolve  MlSS1 6 (FOG'M)  Prog™111  office  and  Boeing  Helicopter  Company  also  may 

The  few  criticisms  which  were  levied  essentially  boiled  down  to  three  areas.  First,  a 
"Tber  ° . peoPle  expected  to  see  more  explicit  human  performance  models,  especially  in 
the  cognitive  area  Functions  such  as  decision  making,  planning,  scheduling,  etewere  not 
emphasized  this  phase  Even  where  present,  such  models  were  often  tfXJSS 

^!fmi^r^fOI!lP0^ltl0n^S!i^U^ component  complicating  their  observation.  Additionally, 
a number  of  attendees  indicated  they  wanted  more  "hard  analysis."  People  wanted  to  know 
specifically  how  MIDAS  could  enable  them  to  find  the  "best”  design  in  terms  of  any 
v m,?asures— h?th  .quantitative  and  qualitative.  They  also  wanted  to  get  to  a 
bottom  line  in  terms  of  mission  success,  etc.  While  "bottom  line"  aspects  have  never 
been  a particular  focus  of  MIDAS,  significant  room  does  exist  for  us  to  improve  the 
analysis  capabilities  included  for  design  evaluation.  Finally,  a number  of  folks  dug  deeply 
enough  to  see  that  we  haven  t yet  reached  the  level  of  integration  among  the  CSCIs  which 
is  intimated.  Distinct  equipment  models  exist  on  both  the  graphic  and  symbolic  sides. 

Task  information  needed  by  the  Training  Assessment  CSCI  is  not  contained  in  the  task 
m^tSTKd<ir  *5®  Symbolic  Modelling  CSCI.  Jack  was  only  demonstrated  in  a stand-alone 
mode,  I tie  lack  of  a properly  functioning  simulation  capability  at  the  start  of  the 
demonstrations  certainly  contributed  to  this  criticism.  However,  the  point  was  generally 
accurate.  The  degree  of  integration  among  the  CSCIs  was  not  where  is  should  have  been  at 
that  point  in  the  overall  development— primarily  because  it  is  the  most  difficult  area  of  all  to 
manage* 
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MAN-MACHINE  INTEGRATION  DESIGN  & *”^*3*3  SYSTEM 
(MIDAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 

PHASE  IV: 


SYMBOLIC  OPERATOR  MODEL 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 

This  is  the  Software  Product  Specification  for  the  Sym^cOiwrator  Model  module  of  the 
Man-machine  Integration  Design  and  Analysis  System  (MIDAS). 


1.2  SCOPE  OF  DOCUMENT 

The  material  in  this  document  is  directed  toward  three  categories  of  readers. 

1)  those  who  wish  to  learn  what  the  MIDAS  Symbolic  Operator  Model 
module  does, 

2)  those  who  wish  to  use  the  Symbolic  Operator  Model  software  to  investigate  the 
interactions  between  an  operator  and  a specific  crew  station  design  within  the 
context  of  a given  mission, 

3)  those  who  might  want  to  modify  and  update  the  Symbolic  Operator  Model 
software. 

1.3  PURPOSE  AND  OBJECTIVE  OF  DOCUMENT 

This  document  attempts  to  describe  the  methodologies  used  to  represent  an  object-oriented 
simulation  environment  in  which  crew  member  activities  are  sensitive  to  crew  stehon 
desicn  The  primary  purpose  of  this  document  is  to  present  the  Phase  IV  methodology 
SSntto!  EX,  goTs  and  crew  member  activities  and  their  interact™ t with  other 
models  dunng  a simulation.  This  methodology  provides  a framework  in  which  models  of 
human  performance  may  be  integrated  for  the  purposes  of  evaluating  alternative  crew 
station  designs. 

This  document  presents  the  results  of  work  accomplished  in  Phase  IV.  ^ « “i^rtant  to 
remember  when  using  this  document  that  the  primary  objective  for  symbolic  modeling  m 
Phase  IV  was  to  demonstrate  a general  framework  for  representing  mission  requirements 
and  aew  ^mber  activities  and  how  these  representations  could  interact  with  other  models 
to  provide  a driving  function  for  a simulation  and  task  history  for  later  analysis.  Many _o 
theP functions  and  data  structures  used  were  selected  on  the  basis  °f. how  they  ^nmbuted  to 
achieving  this  objective.  As  is  typical  in  any  rapid  prototyping  environment,  many  of  these 
functions  and  structures  will  not  be  appropriate  in  future  phases.  However,  attempts  have 
been  made  to  present  what  is  available  in  a manner  which  will  support  development  in 

future  phases. 

A majority  of  the  code  developed  in  Phases  I,  II  and  III  was  not  required  to  meet  the 

curreit  objectives  and  not  incorporated  into  Phase  IV.  A slSniJ)ca ^AS^romm  a^d 
however,  does  address  issues  which  remain  major  concerns  of  the  MIDAS  program  and 


Page  A-l 


may  be  useful  in  future  phases.  This  document  does  not  intend  to  supersede  previous 
documentation  which  address  many  issues  not  addressed  in  Phase  IV,  especially  in  the 
areas  of  decision  modeling.  In  many  cases,  techniques  developed  in  previous  phases  may 
be  incorporated  in  future  work  with  little  or  no  modifications. 

2.0  RELATED  DOCUMENTS 

2.1  APPLICABLE  DOCUMENTS 

James  Allen,  Maintaining  Knowledge  about  Temporal  Intervals",  Communications  of  the 
ACM  26  (11),  832-843,  1983. 

Army-NASA  Aircrew! Aircraft  Integration  Program  (A3I)  Software  Detailed  Design 
Document:  Phase  III,  Contractor  Report  177557,  NASA  Ames  Research  Center,  Moffett 
Field,  California  94035-1000,  June  1990. 

Symbolics  Genera  7.2  Documentation,  Symbolics  Publication  Number  999079, 
Symbolics,  Inc.,  Cambridge,  Massachusetts,  1988. 

Development  of  an  Advanced  Task  Analysis  Methodology  and  Demonstration  for  Army- 
NASA  Air  crew! Aircraft  Integration,  BBN  Laboratories,  NASA  Contract  No.  NAS2- 
12035. 

Boff,  Kenneth  R.  and  Lincoln,  Janet  E.,  Engineering  Data  Compendium,  Human 
Perception  and  Performance,  Harry  G.  Armstrong  Aerospace  Medical  Research 
Laboratory,  Wright-Patterson  Air  Force  Base,  Ohio,  1988. 

Operator's  Manual  for  Army  AH -64A  Helicopter,  TM  55-1520-238-10,  Headquarters, 
Department  of  the  Army,  28  June  1984. 

A Computer  Analysis  to  Predict  Crew  Workload  During  LHX  Scout-Attack  Missions, 
Anacapa  Sciences,  Inc.  October  1984 

2.2  INFORMATION  DOCUMENTS 

A Comprehensive  Task  Analysis  of  the  AH -64  Mission  with  Crew  Workload  Estimates 
and  Preliminary  Decision  Rules  for  Developing  an  AH -64  Workload  Prediction  Model, 
Anacapa  Sciences,  Inc.,  October  1986. 

Sacerdoti,  E.D.,  The  Non-linear  Nature  of  Plans,  Advance  Papers  of  IJCAI-1975. 

Tbilisi,  USSR. 

Stefik,  M.J.,  Planning  with  Constraints,  Artificial  Intelligence,  16,  pp  111-140.  1981. 

Sussman,  G.J.,  A Computational  Model  of  Skill  Acquisition,  Ph.D.  Thesis, 
Massachusetts  Institute  of  Technology,  Cambridge,  MA.,  1973. 

3.0  CONCEPT 

3.1  DEFINITION  OF  SOFTWARE 

3.1.1  Purpose  and  Scope 
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The  development  of  a dynamic  and  interactive  analysis  methodology  for  investigating 
wriralmen^^ignfmssion  and  crew  interactions  simulation 

thaTpilots  bring  to  task  performance,  and  to  provide  the  flexibility  descn^  atove,  this 
module  employs  two  interacting  perspectives  to  guide  the  mission  decompos 

. The  first  perspective  views  the  mission  from  the  aircrew’s  goal  structure  in  mission 
performance. 

. The  second  DersDective  is  that  of  the  software  implementation  architecture  providing  an 
objSf-Sd Mentation  for  the  fundamental  task  units  which  are  denved  from  the 

system  design  style. 

task  performance  time,  performance  load,  models  of  pilot  s decision  and  ^sp^  „ . , 
characteristics  and  mission  specific  doctrine  pertaining  to  task  performance, 
provides  for  the  integration  of  models  of  the  environment,  equipment  design,  mission 
crew  as  the  forcing  function  for  a mission  simulation. 

The  code  developed  in  Phase  IV  has  probably  not  progressed  fa-  enough  to  attempt 

methods  and  structure  were  at  various  stages  of  development  at  the  end  of  Phase  IV, 
recommended  that  the  basic  approach  be  reviewed  and  the  requirements  for  a genera 
framework  be  clearly  stated  before  additional  development  or  modification  is  attempted. 

3.1.2  Goals  and  Objectives 

Goals  for  the  Symbolic  Operator  Model  in  Phase  IV  included  development  of: 

. A domain  independent  representational  framework  to  which  domain-specific  details 
could  be  added  without  writing  system  level  code. 

. Operator  activities  using  multiple  levels  of  abstraetiOT  M^to^tlwel  defined 
in  terms  of  primitive  actions  specified  by  equipment  component  functions. 

. Explicit  representations  of  mission  specific,  design-independent  goals. 
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• Mechanisms  for  function  allocation  based  on  context. 

• An  integration  framework  for  utilizing  anthropometric,  loading  and  scheduling 
models. 

Objectives  for  Phase  IV  included: 

• Demonstration  of  methods  for  representing  key  attributes  of  goals  and  activities  as 
instance  variables  instead  of  determining  attributes  by  inheritance  from  flavor 
components.  This  establishes  an  architecture  which  permits  a wide  range  of  user 
interfaces  and  editing  mechanisms. 

• Development  of  basic  attributes  for  goal  representation  as  opposed  to  development 
of  detailed  data  structures  for  future  phases.  Although  the  structures  for  goal 
representation  used  in  Phase  IV  will  need  to  be  modified  or  reimplemented,  the 
mechanisms  developed  provide  a framework  for  specifying  future  goal 
representation  requirements. 

• Demonstration  of  context-dependent  methods  for  agent  allocation.  This  objective 
focussed  on  the  control  of  when  the  agent  allocation  functions  would  be  evaluated: 
in  the  definition,  initialization,  or  simulation  phases. 

• Demonstration  of  integrating  Jack  commands  into  the  activities.  Methods  for  the 
automatic  generation  of  Jack  commands  were  demonstrated  in  Phase  IV. 

3.1.3  Description 

The  Symbolic  Operator  Model  provides  a model-based  simulation  mechanism  for 
representing  an  operator’s  interaction  with  complex  crew  station  design  in  an  attempt  to 
satisfy  mission  requirements.  The  mechanism  provides  a design  independent 
representation  of  mission  requirements  and  generates  a time-ordered  set  of  operator 
activities.  The  system  is  unique  in  its  ability  to  represent  environmental  context  sensitivity 
in  computing  performance  and  load  information. 

3.2  USER  DEFINITION 

The  user  of  the  Symbolic  Operator  Model  is  considered  to  be  a member  of  the  design  team 
interested  in  using  the  Symbolic  Operator  Model  to  represent  simulated  operator  activities 
to  evaluate  the  design  of  a crewstation. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

A requirement  for  the  MIDAS  simulation  capability  is  that  changes  in  mission  requirements 
or  crew  station  design  affect  mission  performance  and  operator  activity  load.  Mission 
requirements  are  represented  as  goals  and  provide  performance  metrics  suitable  for 
comparing  alternative  crew  station  designs.  Operator  activities  control  the  interaction  of 
operator  models  with  equipment  system  models  and  provide  the  Task  Loading  Model  with 
the  data  necesspy  for  calculating  the  load  values  representing  the  "cost"  associated  with 
performing  a given  set  of  activities.  Operator  activities  also  provide  a means  of  integrating 
other  components,  such  as  the  anthropometric  model,  into  the  simulation  as  well  as 
generating  data  necessary  for  graphical  simulation  displays. 
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A major  architecture  revision  in  Phase  IV  was  the  explicit  j^ntation  ™ 

P • c oitiamati v#*  rnpw  station  designs  since  it  proved  difficult  t 


simulation  analysis. 

entering  a navigation  egress  waypoint  during  the  ingress  segment. 

Another  side  effect  of  having  a single  hierarchical  decomposition  i^t  ^Mgem  ^ ^ ^ 
behaviors  associated  with  responding  to  the.e"\1^™® Jdeled’as  either  separate  segments 
SS S^^tST^h  «Snents  with  dramatic  effect  on  the  computational 
tractability  of  the  task  network. 

The  structures  developed  and  implemented  in  Phase  IV  address  there  problems while 

mmssmsm* 

created  during  post-simulation  processing  to  aid  analysis. 

4.0  REQUIREMENTS 

4 1 REQUIREMENTS  APPROACH  AND  TRADEOFFS 

Z&SSSSSE^B&SS 

processes.  The  requirements  for  the  Symbolic  Operator  Model  include. 

. Fnvirnnmental  models  including  representation  of  terrain  elevation,  simple  features 
SS2SSJ-  friendly  and  hostile  vehicles  and  systems. 
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• Aircraft  systems  models  to  represent  the  basic  aircraft  and  component  systems  at 
adjustable  levels  of  detail. 

• A mission  model  which  is  the  representation  of  actual  mission  requirements  and 
associated  doctrine  and  procedures  as  normally  presented  in  a operations  briefing. 
It  is  not  a representation  of  the  execution  of  a mission  but  rather  a model  which 
helps  drive  a mission  simulation  and  the  standard  by  which  the  results  of  a 
simulation  are  evaluated. 


• Crew  models  to  generate  performance  capabilities  and  task  decomposition  structures 
that  are  sensitive  to  context  and  crew  station  design. 

The  Symbolic  Operator  Model  is  built  upon  the  premise  that  the  design  of  a crew  station 
must  be  evaluated  within  the  contexts  in  which  it  will  be  operating.  It  is  necessary  for  the 
designer  to  be  able  to  vary  characteristics  of  the  pilot,  mission  and  environment  in  order  to 
provide  the  necessary  contexts  needed  to  evaluate  design  alternatives. 

The  MIDAS  system  contains  a tick-based  simulator  which  provides  a discrete  time  model- 
based  simulation  of  the  interactions  of  the  operator  (agent),  equipment  and  environment 
The  simulator  has  the  form  of  a tree  consisting  of  objects.  Each  object  in  it  has  a parent  (or 
is  the  top-level  object)  and  each  object  has  component  objects.  At  each  beat  of  a driver 
clock,  a tick  message  is  sent  to  the  top-level  object  (in  this  case,  the  MIDAS  simulator).  It 
performs  whatever  procedures  it  has  been  programmed  to  carry  out  during  a tick,  then 
passes  the  tick  to  each  of  its  component  objects.  They  carry  out  their  tick  procedures,  then 
pass  the  tick  downward,  and  so  on.  This  approach  was  taken  to  meet  the  design  goal  of 
modularity.  Anything  that  handles  a tick  message  may  be  added  to  a list  of  component 
objects.  This  architecture  allows  for  great  flexibility  and  modularity  in  building  simulations. 

4.2  EXTERNAL  INTERFACE  REQUIREMENTS 

The  Symbolics  Operator  Model  is  required  to  communicate  with  external  models  through 
the  Communications  module,  which  is  described  in  Annex  J. 

4.3  HARDWARE  ENVIRONMENT 

The  Symbolic  Operator  Model  software  runs  on  the  Symbolics  3600  series  of  workstations 
with  22.5  MB  of  RAM  and  338  and  550  MB  hard  drives  and  an  Ethernet  Controller  and 
Transceiver. 

4.4  SOFTWARE  ENVIRONMENT 

The  Symbolic  Operator  Model  software  is  written  under  the  Genera  7.2  operating  system 
with  extensive  use  of  Flavors  for  object-oriented  programming.  TCP/IP  software 
communication  software  is  required  for  integration  with  other  modules  of  MIDAS 

A decision  was  made  in  the  beginning  of  the  phase  to  continue  to  use  the  Symbolics  Flavor 
System  for  object-oriented  programming  for  Phase  IV  development.  However,  the 
general  acceptance  of  the  Common  Lisp  Object  System  (CLOS)  and  the  release  of 
Symbolics  Genera  8.0  with  CLOS  indicate  that  future  development  should  probably  be 
developed  within  CLOS.  It  was  apparent  throughout  Phase  IV  that  the  Flavor  System  was 
probably  not  appropriate  for  long  term  development  and  that  a transition  to  a more  standard 
Common  Lisp  and  CLOS  should  be  accomplished.  It  is  clear,  in  retrospect,  that  the 
decision  to  stay  with  the  Flavor  System  for  Phase  IV  was  correct  since  time  and  resources 
would  not  have  permitted  a major  transition.  Rather  than  produce  production/deliverable 
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quality  code,  the  emphasis  in  Phase  IV  was  directed  at  exploring  hasiCcon^pKincludmg 
die  introduction  of  explicit  goals,  the  use  of  templates  for  acuities  and (goals, - 
integration  of  external  components  such  as  the  anthropometric  model.  . 

codedeveloped  in  Phase  IV  is  not  how  a specific  so  ution  was  impler^nt^butrathCT  die 

identification  of  the  problems.  For  example,  a temPlate^h^^{"^Wt7s^Stha^ 
will  orobablv  be  significantly  different  than  the  approach  use  in  Phase  IV  but  it  is  clear  that 
SfiSon  wdl  need  to  be  made  to  how  values  are  passed  from  templates  to  goals  or 

activities. 


4.5  REQUIREMENTS  SPECIFICATION 


The  requirements  for  the  Symbolic  Operator  Model  are: 


♦ Symbolic  representation  of  the  environment, 
vehicles  and  operators  to  the  extent  that  they 


including  terrain  models  and  other 
affect  the  primary  operator's  activities. 


• Integration  with  the  representations  of  the  operator's  vehicle,  crew  station  and 
associated  subsystems. 

• Symbolic  representation  of  a mission  for  a complex,  rotary  wing  aircraft. 


Symbolic  representation  of  an  operator  performing  tasks  in  a complex  crew  station 
in  which  the  operator's  activities  are  sensitive  to  the  design  of  the  crew  station. 


• Symbolic  representation  of  concurrent  activities  within  constraints  of  resources  and 
load  factors. 


. Activities  generated  by  mission  requirements  sensitive  to  context  and  not  limited  to 
planned  sequence. 

. Activity  sequence,  duration  and  load  sensitive  to  human  performance  models  and 
equipment  design. 

• Integration  of  the  symbolic  operator  model  with  anthropometric,  loading  and 
scheduling  models. 

• Integration  of  symbolic  models  with  graphical  models  existing  on  other  hardware 
by  means  of  an  independent  communications  program. 


• Provide  a task  decomposition  and  a simulation  driving 
dependent  on  the  interactions  of  the  mission,  operator, 
models 


function  which  are 
vehicle  and  environmental 


• Task  allocation  based  on  context. 

. Mechanisms  for  adding  domain  specific  functions  and  attributes  without  requiring 
system  level  code  development. 

* Mechanisms  for  crew  interaction. 
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5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

5.1.1  Architectural  Design  Description 

The  symbolic  modeling  components  of  a MIDAS  simulation  are  developed  in  three  phases: 
definition,  initialization  and  simulation  phases.  These  phases  must  be  performed  in 
sequence.  If  a definition  for  a default  value  is  changed,  the  initialization  and  simulation 
phases  should  be  redone.  These  three  phases  are  illustrated  in  Figure  1. 


Definition 

Phase 


Figure  1.  Symbolic  Modeling  Development 


5.1. 1.1  Environment  Modeling 
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Although  not  a requirement,  defining  the  environment  model  is  typicaUy  the  fiist  step 
developing  a MIDAS  simulation.  The  environment  is  modeled  using  a digital  elevation 
terrain  model  (DEM)  and  a collection  of  static,  abstract  and  active  objects. 

5. 1.1. 1.1  Terrain 

The  DEM  model  is  defined  as  an  object  with  an  area  specified  in  reference  to  the  Universal 
Transverse  Mercator  (UTM)  coordinate  system  and  provides  elevation  in  feet  above  sea 
level  (MSL)  for  given  x and  y values.  The  DEM  model  also  provides  a oansfonnation 
between  the  UTM  coordinates  and  an  internal  coordinate  system  of  x and  y (m  feet)  used 
for  graphics  and  aerodynamic  modelling.  A method  of  the  DEM  object  is  a predicate 
funSon  which  determines  if  line  of  sight  is  possible  between  two  3-D  pomts  and  is  used  to 
determine  line  of  sight  between  environmental  objects.  The  actual  structure  of  the  DEM 
object  may  be  varied  as  desired  and  terrain  elevation  values  may  be  stored  widun  die 
structure  itself  or  accessed  with  foreign  function  calls  toute™^m^^P™^a^eFM 
Silicon  Graphics.  Simulation  objects  should  reference  the  terrain  model  by  ^e  same  DEM 
methods  regardless  of  whether  the  elevation  values  are  stored  within  the  DEM  object  or 
accessed  by  DEM  methods  from  the  Silicon  Graphics  machines 

5.1. 1.1.2  Static  Objects 

Static  objects  are  used  to  represent  environmental  features  whose  characteristics  (i.e. 
location,  shape,  etc)  remain  constant  during  a simulation.  Roads  and  buildings  are 
examples  of  potential  static  objects  for  a MIDAS  simulation.  Static  objects  require 
definition  of  their  location  in  terms  of  x,  y,  and  z values.  Additional  object  features  may  be 
added  as  required  by  the  domain. 

5.1.1. 1.3  Abstract  Objects 

Abstract  objects  are  features  of  the  environment,  such  as  phase  lines  and  operations  orders, 
which  do  not  necessarily  physically  exist  and  normally  will  not  have  a graphical 
representation  in  the  Views  module.  Abstract  objects  may  be  implemented  as  actual  object 
instances  or  implied  by  reference.  When  implied  by  reference,  the  object  should  be 
accessible  by  a clearly  defined  method  (e.g.  the  method  "(phase-lines)"  provided  access  to 
the  phase  lines  defined  in  Phase  IV  although  they  are  actually  implemented  as  values  in  a 
list  instead  of  as  an  object). 

Two  abstract  objects  used  in  a MIDAS  simulation  include  the  scenario  and  crew-mission 
objects. 

5. 1.1. 1.3.1  Scenario 

The  scenario  object  is  a data  structure  used  to  store  initialization  values  and  lope  for 
controlling  the  simulation  during  execution.  This  abstract  object  specifies  the  lninal 
position  of  physical  objects  in  the  domain  and  provides  a mechanism  for  controlling 
external  events  during  the  simulation. 

Scenario  instances  are  created  during  the  simulation  initialization  and  a new  instance  is 
created  for  each  simulation  run. 
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Parameters  for  scenario  instances  include  the  following  (full  structural  details  provided  in 
later  sections): 


sim-name 

sim-id 

envir-context 


world-objects 

e-transmissions 


initial-time 

current-time 

current-tick 

control-events 

int-cued-events 

int-acdve-events 

int-terminated-events 


=>  a string  representing  the  scenario  name 
=>  an  unique  id  for  each  instance 
=>  list  of  <environmental-parameter>  <value>  pairs 
For  example: 

:envir-context 

'Ophase-lines 

;;;  A phase  line  called  RED  from  grid  location 
;;;  48008400  to  48007900  and  controlled  by 
;;;  D-1/26CAV 

((RED  48008400  48007900  D-1/26CAV) 

;;  A phase  line  called  BLUE  from  grid  location 
;;;  50008400  to  50007900  and  controlled  by 
;;;  B-1/26CAV 

(BLUE  50008400  50007900  B-1/26CAV)) 
:acps  ;;;  Air  Control  Points  (ACP) 

;;;  An  air  control  point  called  ACP-DELTA 
;;;  located  at  grid  48208030  and  controlled 
;;;  by  the  tactical  operations  center  (TOC) 
((ACP-DELTA  48208030  TOC)) 

:farp  47008000  ;;;  Forward  Arm  and  Refuel  Point 
(FARP) 

:flot  ;;;  Forward  Line  Own  Troops  (FLOT) 
’(54008400  54007900)) 

=>  used  to  designate  active  world  objects 
=>  time-tagged  electronic  transmissions. 

Transmitting  a mission  causes  the  message  to 
be  posted  here  with  frequency  and  time  tags. 

=>  indicates  what  scenario  time  should  be  at  tick  0. 

=>  provides  central  location  for  current  time  representation 
=>  provides  central  location  for  current  tick  representation 
=>  events  used  to  control  simulation  flow 
=>  identifies  when  events  should  happen  during  the  simulation 
=>  scenario  controlled  events  in  progress 
=>  history  of  terminated  scenario  events 


5.1. 1.1. 3. 2 Crew-mission 


The  crew-mission  object  is  a data  structure  used  to  provide  information  normally  provided 
by  operations  orders,  standard  operating  procedures  (SOPs),  and  doctrine.  Callsigns  and 
assigned  frequencies,  required  reports,  and  planned  route  waypoints  are  among  the 
information  provided  by  the  crew-mission.  The  actual  representation  of  the  planned  means 
of  executing  the  mission  is  the  result  of  integrating  the  crew-mission  information  with  the 
procedures  of  the  mission  goal  hierarchy  into  a task  hierarchy.  This  task  hierarchy  is 
created  by  instantiating  subgoal  templates  with  arguments  supplied  by  the  associated  default 
goal  templates  and  crew-mission  details.  These  subgoal  templates  are  used  during  the 
simulation  to  create  the  actual  instances  of  goals.  It  should  be  noted  that  one  subgoal 
template  may  create  many  different  instances  of  a goal  since  values  for  the  goal's  instance 
variables  may  be  determined  at  run-time  according  to  context.  For  example,  a goal  for 
"turn-to-assigned-heading"  will  have  different  values  for  the  heading  to  achieve.  In  fact, 
even  the  agent  allocated  the  goal  may  vary  during  since  the  subgoal  template  may  specify 
that  the  goal  should  be  allocated  to  the  crew  member  currently  flying.  At  the  time  the  goal 
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is  instantiated  during  the  simulation  (i.e.  as  result  of  an  radio  cdl  assi^inganewheading) 
the  value  for  the  agent-allocation  parameter  is  determined  simply  by  which  agent  is 
currently  flying. 

Crew-mission  instances  are  created  during  the  simulation  initialization  and  a new  instance  is 
SSto each  simulation  ran.  Parameters  for  crew-mission  instances  rnclude  the 
following  (full  structural  details  provided  m later  sections): 


mission-name 

assigned-crew 

mission-role 

callsign 

goal-net 

point-of-departure 


waypoints 

routes 

planned-route 

default-comm-assign 

ceoi 

comm-req 


=>  string  representing  mission  name 

=>  instances  of  agents  , r_  n 

=>  role  symbol  (e.g.  TM1-AC1  = Aircraft  1 of  Team  1) 

=>  assigned  communication  callsign  (e.g.  Y5T4o) 

=>  specifies  top-level  goal  node  for  planned  mission  goals 
=>  grid  location  of  mission’s  point  of  departure. 

Note-  since  the  simulation  may  represent  only  a segment 
of  a mission,  the  aircraft  may  never  actually  be  at  the 
point  of  departure  during  the  simulation. 

=>  preplanned  navigation  waypoints 
=>  preplanned  routes 

=>  preplanned  route  with  altitude  information 
=>  planned  communication  task  assignments 
=>  Communications  Electronic  Operating  Instructions  (CEU1) 

Callsigns,  frequencies  and  communications  nets. 

=>  required  communication  tasks  (e.g.  report-crossing-phase-hnes) 


Information  that  is  typically  available  to  a crew  member  performing  a mission  in  the  real 
ItSXSto  the  agent  model  representing  that  crew  member  by  the  crew- 
mission  object.  Information  that  is  necessary  for  a simulation  and  does  not  correspond  to 
real  life  missions  is  normally  provided  by  the  scenario  object 


5. 1.1.2  Active  Objects 

Active  obiects  represent  items  which  can  respond  to  changes  in  the  environment  or  have 
internally  defined  behavior  resulting  in  state  changes  during  the  simulanort  Active  objects 
may  represent  functional  systems,  agents,  or  complex  man-machine  systems. 

Functionally  active  objects,  such  as  electro-mechanical  systems,  may  be  represented  by 
defining  a "tick"  function  to  determine  changes  in  the  object  s state  for  ca{* 
time  dunng  the  simulation.  This  function  is  associated  directly  with  the  object  and  is 
Suared See  5 time  the  object  is  sent  a "tick”  message.  Inputs  to  this  function  may  be 
state  parameters  of  other  objects  or  parameters  changed  by  agents  by  means  of  activities. 

5.1. 1.2.1  Agents 

Human  operators  are  represented  as  agents  whose  behaviors  are  defined  in  terns  of 
activities  P Activities  are  implemented  as  instances  of  action  using  an  object-oriented  data 
structure  and  may  be  predefined  as  a set  of  actions  in  a partially  ordered  sequence 
(equivalent  to  a pre-defined  script)  or  driven  by  goal  directed  behavior  AcG^ties  provtde  a 
mechanism  for  an  agent  to  interact  with  equipment  and  the  environment  and  will  be 
discussed  in  detail  in  later  sections. 

rv.ir.nW  man-machine  systems,  such  as  other  vehicles  and  radar  sites,  may  be  represented 

with  explicit  agent  models  linked  to  functional  ^ipment  fSoH^ 

objects  in  which  the  crew  member’s  control  of  the  system  is  implicit  in  the  tick  tunctio 
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°r  b)L^ activities  *°  a functional  object  It  is  important  that  a MIDAS  user  is  not 
^ onc  representational  method  and  the  decision  of  which  technique  is  used 

s ouki  be  determined  by  the  domain  requirements.  In  previous  phases,  convoy  vehicles 
and  other  aircraft  were  adequately  modeled  with  one  object  structure  (as  opposed  to  a 

ob£ckt dnVCT  °bjeCt  interacting  with  a "tmck"  obje«>  ^ adding  activities  tothe  vehicle 

Agents  interact  with  their  environment  (and  other  agents)  by  means  of  activities.  Each 
agent  is  permitted  to  perform  concurrent  and  overlapping  activities  when  resources  and 
loading  limitations  permit.  Agent  resources  in  Phase  IV  consisted  of  the  left  and  right 

S?fin^d  referu,nCe  .“defined  by  a 3D  point  in  space.  These  resources  were 
defined  to  be  compatible  with  Jack,  the  anthropometric  model  provided  by  the  University 
of  Pennsylvania.  Load  limitations  were  based  on  a system  which  describes  load  in  terms 

V cognitive  and  motor  (V  ACM)  components.  Each  action  has  associated 

V ACM  components  and  an  agent  s VACM  load  components  are  computed  by  the  Task 
Leading  Model  based  on  the  agent’s  activities.  The  Task  Loading  Model  is  presented  in 

SL^nnCX  S'  An  !§' *!!*  Isjd,ovred  to  Perform  concurrent  activities  if  resource  and  load 
limitations  are  not  excelled  and  within  the  physical  limitations  of  the  crew  station  design 
Extensive  mechanisms  have  been  implemented  and  demonstrated  in  Phase  IV  to  provide  a 
framework  which  insures  agent  activities  are  consistent  with  crew  station  design  This  is 

specif,c  mMts  wi*  “mpoKn‘ fmcAons  deflned 

Two  mechanisms  were  developed  in  Phase  IV  to  control  the  selection  and  sequencing  of  an 
agents  activities.  The  first  involved  a generalization  of  mechanisms  developed  in  previous 
phases  for  controlling  activities  from  within  other  activity  structures  and  is  a logical 
progression  of  work  done  in  those  phases.  The  second  mechanism  involved  controlling 
n vfp  icu  y rePresentinS  goals.  Goals  were  introduced  for  multiple  purposes8 
dSn nK 5?  „a  fCanf  °f  more  cle“lycomparing  separate  designs  since  actiSSs  of  one 
h8i  ft  d , easi  y ™ap  ,nto  1,16  altemative  design.  Second,  goals  provide  a means 
of  delaying  until  run-time  the  construction  of  a task  hierarchically  (or  multiple  hierarchic) 
This  allows  the  use  of  context  to  reduce  search  space.  This  appears  especially  nSSa^  in 
domains  where  extensive  contingency  behavior  is  required.  In  domains  where  the 
sequencing  of  an  operator's  actions  is  generally  predictable,  the  use  of  goals  may  not 

daaifin  a ^^010^  advantage’  The  use  of  8oals  control  activities  is  prerented  in 


An  AGENT'S  instance  variables  include  the  following: 
(see  later  sections  for  full  structure  details) 


->  unique  id  made  by  "AGENT"+  gensymed  number 
(e.g.  AGENT-323 ) 

=>  function  role  (e.g.  PILOT  or  CPG) 


E-R-GOALS: 

GOALS: 

TERMINAL-GOALS: 

GOAL-FILE: 


=>  list  of  instances  of  event-response  goals 
=>  list  of  instances  of  decomposable  goals 
=>  list  of  instances  of  terminal  goals 

=>  history  of  goals  from  the  above  lists  which  have  terminated 


ACTIVITIES: 

CUED-ACTIVITIES: 

TERMINAL-ACTS: 

ACT-FILE: 

INT-TEMP-VACP: 


->  list  of  instances  of  current  decomposable  activities 
=>  list  of  activity  instances  waiting  to  start  - (activity  wait  list) 
->  list  of  instances  of  terminal  activities. 

=>  list  of  terminated  activity  instances 
=>  used  internally  to  allocate  resources 
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RESOURCE-COMMIT:  =>  record  of  resource  allocations 

V ACP-HISTORY : =>  time-tagged  history  list 

=====  The  remaining  variables  concern  anthropometric  modeling 

(see  Annex  G for  details) 


Separate  instance  variables  for  both  right  and  left  hand  for. 


<RH  or  LH>-REACH-ID: 
<RH  or  LH>-REACH-SITE: 
<RH  or  LH>-X: 

<RH  or  LH>-Y : 

<RH  or  LH>-Z: 

<RH  or  LH>-STATUS: 
<RH  or  LH>-LAST-SITE: 


=>  reach  type 
=>  predefined  site 
=>  x location  of  hand 
=>  y location  of  hand 
=>  z location  of  hand 

=>  estimated  reach  time  remaining  - 0 = hand  at  site 
=>  site  reach  initiated  from 


HEAD-TURN-ID: 

SITE-TO-LOOK-AT: 

HEAD-YAW: 

HEAD-PITCH: 

LAST-SITE-LOOKED-AT: 


=>  specialization  of  reach-id 

=>  predefined  site 

=>  direction  head  is  turned 

=>  angle  head  is  tilted  up  or  down 

=>  last  predefined  site  to  which  the  head  was  fixed 


5.1. 1.2.2  Activities 

Activity  structures  are  used  to  describe  agent  behavior.  An  activity  is  an  object-oriented 
data  structure  which  represents  a specific  instance  of  an  agent’s  action.  Agent  behavior 
may  be  represented  with  many  levels  of  abstractions  since  each  activity  may  be  _ 
decomposed  into  a hierarchical  network  of  subactivities.  Activity  structures  are  either 
decomposable  or  terminal.  Decomposable  activities  have  subactivities  which  m turn  may 
be  decomposable  or  terminal.  Terminal  activity  structures,  the  lowest  level  of  activity 
decomposition,  provide  mechanisms  for  integrating  an  agent  s action  with  other  objects. 
The  mechanisms  enable  a terminal  activity  to  change  either  the  agent  s or  other  object  s 
state  by  executing  the  component  functions  describe  in  the  activity  s structure.  1 hese 
component  functions  are  defined  based  on  equipment  system  design  and  provide  the  means 
of  making  an  agent's  behavior  sensitive  to  design.  Component  functions  are  described  in 
Annex  D. 

Each  activity  has  a discrete  temporal  extent  with  associated  times  for  initiation,  start,  and 
termination.  Activity  initiation  time  represents  the  earliest  time  an  activity  could  have 
started.  Initiation  time  may  vary  from  stait  time  due  to  delay  in  starting  an  activity  because 
of  resource  constraints.  Activities  which  have  been  initiated  but  not  yet  started  are 
classified  as  rned-activities.  Activities  which  have  started  are  classified  as  a£iud£  activities. 
Termination  time  represents  when  an  activity  finished  which  could  result  from  completion 
or  interruption.  Since  activities  have  discrete  temporal  extent,  an  action  which  is 
interrupted  and  resumed  is  represented  with  two  activity  object  instances  with  the 
relationship  between  the  two  expressed  explicitly  in  the  activity  at  the  next  higher  level  of 
abstraction. 


Control  of  activity  initiation,  starting  and  termination  is  accomplished  with  initiation,  start 
and  termination  conditions  defined  in  the  activity  structure.  A number  of  mechanisms  have 
been  provided  which  reduce  the  effort  necessary  to  define  activities  by  allowing  initiation, 
starting  and  termination  conditions  to  be  defined  once  for  a set  of  activities  in  the  structure 
of  the  activity  at  the  next  higher  level  ("parent  activity")  of  abstraction.  These  mechanisms, 
which  include  definitions  of  sequential,  parallel  and  rotational  activities,  are  described  in 
detail  in  later  sections. 
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5.1. 1.2.2. 1 Script-Based  Agent  Behavior 

As  in  previous  phases,  control  of  an  agent  may  be  accomplished  by  providing  the  desired 
control  logic  within  the  structures  of  activities.  This  is  accomplished  by  specifying  an 
activity  as  the  top  node  in  a hierarchy.  This  activity  specifics  subactivities  and  the  temporal 
relations  between  each  of  the  subactivities.  The  temporal  relations  may  be  expressed  as  a 
general  relation  which  applies  to  the  subactivities  in  the  order  that  they  are  specified.  For 
example,  top-level  activity  X is  specified  as  a sequential  activity  and  specifies  subactivities 
A,  B,  and  C.  In  this  example,  subactivity  A will  start  when  the  activity  X starts. 
Subactivity  B will  start  after  subactivity  A has  been  completed  and  C will  follow  B 
likewise.  Rotation  activities  would  work  in  a similar  manner.  Parallel  activities  actually  are 
implemented  in  a different  manner  than  in  previous  phases.  In  this  phase,  a default  logic 
was  implemented  in  which  all  subactivities  will  be  started  unless  otherwise  constrained  (i.e. 
by  a sequential  relation  specification,  resource  constraints,  etc).  As  a result,  an  activity 
specified  as  a parallel  activity  simply  allows  each  subactivity  to  start.  The  main  difference 
between  this  approach  and  that  taken  in  previous  phases  concerns  the  specification  of  what 
had  been  called  parallel-stop  activities.  In  previous  phases,  a parallel  activity  would 
continue  to  be  active  until  all  subactivities  were  completed.  Parallel-stop  activities, 
however,  stop  as  soon  as  any  subactivity  completed.  This  mechanism  was  provided  by 
specifying  parallel-stop  as  a flavor  component  of  the  activity.  In  this  phase,  however, 
parallel-stop  activities  are  specified  by  including  in  the  termination  conditions  a test  which 
returns  "true”  if  any  subactivity  is  completed.  The  differences  between  parallel  and 
parallel-stop  activities  in  this  phase  from  those  in  other  phases  are  related  to  implementation 
and  conceptually  remain  the  same. 

This  new  default  logic,  however,  provides  a more  complex  and  flexible  mechanism  for 
controlling  subactivities  than  those  provided  previously.  In  previous  phases,  temporal 
relationships  between  the  subactivities  were  limited  to  a single  term  which  was  applied  to 
all  subactivities.  Using  the  new  default  logic,  it  is  now  possible  to  express  temporal 
relations  (actually  any  type  of  constraint)  which  can  be  applied  to  a subset  of  the 
subactivities  (i.e.  C follows  A independent  of  the  status  of  B).  These  temporal  relations 
can  be  point-based  or  interval-based  (using  James  Allen's  temporal  intervals).  Also,  non- 
temporal constraints,  such  as  resource  constraints,  can  be  used  to  control  the  sequencing  of 
activities.  This  basic  mechanism  provides  a means  of  implementing  any  type  of  script- 
based  scenario. 

5. 1.1.2. 2. 2 Goal-Directed  Agent  Behavior 

A major  development  in  Phase  iy  was  the  introduction  of  explicit  goals.  Goals  were 
introduced  as  a means  of  controlling  the  generation  of  top-level  activities  enabling  an  agent 
to  have  multiple  activity  hierarchies.  There  were  many  factors  that  influenced  this  decision. 
First,  activities  are  intended  to  represent  actions  that  the  agent  performs  and  a separate 
structure  was  desired  to  represent  states  that  should  be  achieved  during  a mission.  An 
alternative  approach  would  be  to  use  the  same  activity  structures.  This  raises  problems, 
however,  when  tasks  are  shed  or  neglected.  Additional  problems  arise  during  multi-agent 
interaction  when  a side  effect  of  one  agent's  action  achieves  the  states  that  another  agent  is 
trying  to  achieve.  In  this  case,  the  "goal"  has  been  achieved  but  it  is  unclear  which  agent's 
activity  structure  should  represent  this.  By  explicitly  representing  goals  as  separate 
structures,  it  is  easy  to  represent  shed/neglected  tasks  and,  in  the  multi-agent  example, 
represent  one  agent's  activity  achieving  another  agent's  goal.  Second,  it  seem 
advantageous  to  have  an  representation  that  explicitly  represented  the  standards  since 
alternative  crew  station  designs  may  have  different  activities  for  the  same  mission.  Hence, 
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the  goals  are  intended  to  represent  design  independent  standards  by  which  competing 
designs  could  be  compared. 


Goals  in  Phase  IV  were  represented  in  a manner  similar  to  activities.  A goal  may  have 
subgoals  and  similar  expression  of  temporal  relations  was  provided.  A goal  which  has 
subgoals  specified  is  referred  to  as  a decomposable  goal.  At  the  lowest  level  in  a given 
goal  hierarchy  are  terminal  goals  which  have  no  subgoals  but  specify  states  that  need  to  be 
achieved  or  maintained.  These  states  are  expressed  in  terms  which  are  mappable  to  states 
achieved  by  an  agent's  activities  or  equipment  systems. 


Goals  are  characterized  from  a number  of  perspectives.  First,  as  explained  above,  goals 
are  either  decomposable  or  terminal.  Second,  goals  are  cither  achievement  or  maintenance 
goals.  Third,  goals  are  classified  as  to  whether  they  are  satisfied.  Fourth,  goals  may  be 
either  current,  active  or  inactive.  Fifth,  goals  may  have  cither  abstract  or  explicit  function 
allocation. 


Achievement  goals  specified  a state  (e.g.  "target  detected")  and  the  goal  is  considered 
satisfied  as  soon  as  this  state  is  achieved.  Achievement  goals  are  classified  as  either 
satisfied  or  not-satisfied. 


Maintenance  goals  specify  states  (e.g.  "maintain  heading  135")  which  must  be  maintained 
for  a specified  duration.  Maintenance  goals  are  classified  as:  _ 

currently-satisfied  = state  maintained  but  duration  of  goal  has  not  yet 

been  completed 

not-currently-satisified  = state  not  maintained  but  duration  of  goal  has  not 

yet  been  completed 

satisfied  = state  maintained  and  duration  completed 

not-satisfied  = state  not  maintained  and  duration  completed. 


Goals  are  considered  current  when  it  is  possible  to  test  if  the  desired  state  has  been 
achieved.  A goal  to  "pass  over  point  A at  5000  feet”  can  only  be  tested  at  the  time  the 
aircraft  passes  over  point  A. 


Goals  are  considered  active  when  the  desired  states  of  the  goal  influence  actions  currently 
being  taken.  In  order  to  achieve  "pass  over  point  A at  5000  feet",  some  actions  (i.e. 
establish  descent)  may  need  to  be  achieve  prior  to  the  time  of  passing  A.  The  requirements 
for  satisfying  the  preconditions  of  a goal  are  identified  when  a goal  becomes  active. 
Inactive  goals  are  merely  goals  which  do  not  influence  current  behavior. 


Goals  have  explicit  function  allocation  when  the  goal  identifies  a unique  agent  (e.g.  the 
pilot").  A new  feature  was  introduced  in  Phase  IV  which  enables  goals  to  be  dynamically 
allocated  based  on  context  by  specifying  an  abstract  function  allocation.  An  abstract 
function  allocation  is  accomplished  by  specifying  a set  of  agents  and  a Lisp  font*  to  be 
evaluated  at  run  time  which  will  select  one  of  these  agents  based  on  context.  For  example, 
a goal  could  be  assigned  to  the  crew  (i.e.  the  set  of  pilot  and  cpg)  and  provide  a form  which 
selects  the  crew  member  with  the  lowest  task  requirements. 


During  a simulation,  each  agent  responds  to  a tick  message  in  the  following 
manner 

1.  Update  Current-Time.  . . , , 

2.  Delete  any  goals  no  longer  justified  or  completed  from  active  and  matched  goal 

lists. 

3.  Identify  new  event-response  goals. 

4.  Add  newly  activated  goals  from  the  mission  hierarchy. 
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7.  Identify  matched-goals  with  terminated  or  completed  activities.  Move  goal  from 
matched  goal  list  to  active  goal  list. 

7.  Map  activities  to  goals  in  active  goal  list.  Add  action  to  activity  wait  list. 

8.  Sort  activity  wait  list  based  on  goal  priorities. 

9.  For  each  activity  in  wait  list,  start  activity  when  resource,  load,  and  temporal 
constraints  permit. 

10. Test  termination  conditions  to  each  current  activity  - delete  activity  when  conditions 
indicate. 

11.  Execute  tick  procedure  for  each  active  activity. 

12.  Signal  simulation  executive  cycle  completed. 

In  previous  phases,  relationships  between  activities  were  determined  by  the  details  of  a 
single  hierarchy  and  contingency  behavior  was  represented  as  branches  of  the  hierarchy.  If 
an  event  that  indicated  some  activities  should  be  performed  had  the  potential  of  occurring  at 
different  segments  of  the  mission,  the  activity  for  responding  to  this  event  had  to  either  be 
modeled  as  a separate  preemptive  segment  or  modeled  repetitively  in  each  segment  that  the 
event  potentially  could  occur.  As  a result,  adding  contingency  behavior  to  respond  to 
multiple  events  possibly  occurring  throughout  the  mission  would  produce  an  activity 
hierarchy  that  would  be  hard  to  manage  if  not  intractable. 

The  approach  used  in  Phase  IV  addresses  this  problem  by  allowing  multiple  activity 
hierarchies.  From  a planning  perspective,  the  mechanisms  provide  means  of  resolving 
conflicts  arising  from  resource  competition.  The  approach  does  not  resolve  conflicts 
arising  from  conflicting  state  sequences  such  as  the  Sussman  Anomaly  (Sussman,  1973), 
however,  in  many  situations  this  does  not  present  a problem  since  many  concurrent 
activities  do  not  involve  states  that  intersect.  For  example,  the  interactions  between 
concurrent  activities  of  transmitting  a radio  message  and  updating  a navigation  system 
involves  resources  more  than  equipment  state  dependencies.  Since  resource  commitments 
are  made  at  the  time  an  activity  is  activated,  the  approach  is  in  some  ways  similar  to 
planners  allowing  partial  ordering  (Sacerdoti,  1975)  and  constraint  posting  (Stefik,  1981). 
The  objective  in  Phase  IV  was  not  to  develop  or  implement  an  actual  planner  but  to' 
demonstrate  how  such  systems  could  be  integrated  into  the  simulation.  The  state 
dependencies  of  different  functions  on  a Multi-Function  Display  clearly  indicate  a 
requirement  for  approaches  that  address  the  issues  of  conflicts  between  subgoal  state 
sequences.  It  is  believed  that  such  approaches  could  be  integrated  into  the  framework  of 
the  agent  s tick  method  presented  above. 

5.2  DETAILED  DESIGN 

5.2.1  Detailed  Design  Approach  And  Tradeoffs 

5.2.1. 1 Activity  Definition 

A key  objective  of  the  Symbolic  Operator  Model  is  the  development  of  an  agent  model 
whose  behavior  is  sensitive  to  changes  in  crew  station  design.  In  order  to  achieve  this, 
terminal  activities  are  defined  using  equipment  component  functions  as  the  mechanisms  for 
the  agent  to  interact  with  equipment  models.  Equipment  component  functions  are  defined 
during  the  process  of  developing  an  equipment  model  (see  Annex  D).  These  component 
functions  specify  state  changes  which  should  occur  as  result  of  a specified  action.  By 
defining  terminal  activities  using  these  component  functions,  any  change  in  the  crew  station 
equipment  models  will  have  a direct  effect  on  the  agent's  behavior.  The  state  changes 
resulting  from  executing  an  equipment  component  function  are  specified  in  a manner  that 
clearly  represents  the  duration  of  the  action  and  how  state  is  changed  during  each  "tick"  of 
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time  during  the  action.  Within  an  terminal  activity,  equipment  component  functions  are 
executed  in  sequential  order. 


In  order  to  provide  an  activity  modeling  environment  in  which  activity  definitions  can  be 
described  and  changed  without  encountering  the  problems  associated  with  changing 
mentioned  previously  concerning  flavor  definitions,  a mechanism  was  developed  in  Phase 
IV  to  aid  the  process  of  activity  definition.  The  key  feature  is  the  use  of  template  data 
structures  instead  of  flavor  definitions.  Templates  are  used  to  define  default  activity 
parameters  and  specify  global  changes  to  an  activity  class  definition,  for  specialization  of 
both  class  and  instances  to  provide  exceptions  to  activity  class  default  values,  and  to  make 
activity  instances  context  sensitive. 


Templates  are  provided  at  two  levels: 

• First,  default  templates  which  provide  a definition  of  a general  activity  class. 

• Second,  subact  templates  which  provide  specialization  of  default  activity  templates 
based  on  the  context  of  the  mission  and  the  environment. 


Default  activity  templates  are  defined  by  the  users  during  a pre-simulation  activity  definition 
phase  which  must  be  accomplished  after  the  equipment  models  have  been  defined.  Subact 
templates  are  later  automatically  generated  during  a pre-simulation  activity  initialization 
which  must  be  performed  after  activity  definition  and  mission  definition  phases  have  been 
completed. 


If  a user  wants  to  change  a parameter  value  associated  with  all  instances  of  a specific 
activity  class,  the  specifications  in  the  default  activity  template  should  be  changed.  The 
default  activity  templates  for  decomposable  activities  indicate  how  sub-act  templates  should 
be  created  for  its  subactivities.  If  the  subactivities  are  to  be  created  with  the  values 
specified  by  a default  activity  template,  the  user  need  only  specify  the  name  of  the  activity 
and  a sub-act  template  will  be  created  using  the  default  values.  If  the  user,  however,  wants 
to  make  changes  to  the  default  values  when  certain  conditions  apply  or  for  a subset  of  an 
activity  class,  the  user  can  specify  conditions  in  the  parent  default  activity  template  that  one 
its  subactivities  should  be  created  from  a template  with  values  other  than  the  defaults  (see 
later  sections  for  details).  These  mechanisms  provide  logic  for  controlling  inheritance  both 
globally  and  locally.  The  alternative  approach  of  defining  separate  activity  templates  for 
both  global  and  local  definitions  would  require  producing  a significant  larger  number  of 
user  defined  templates  and  possibly  obscure  whether  the  intention  was  to  redefine  the  class 
or  just  define  a subset  with  minor  variations. 


Default  activity  parameters  are  defined  using  DEFAULT- ACTIVITY-^TCMPLATES. 

These  templates  are  instances  of  the  flavor  DEFAULT-ACT1VITY-TCMPLATE  and  the 
desired  activity  parameters  are  specified  as  values  of  instance  variables.  These  activity 
template  instances  define  a class  definition  for  typical  activities.  Unlike  previous  phases  in 
which  changes  in  the  definition  of  a class  of  activities  required  recompiling  flavor 
definitions,  class  definitions  can  be  modes.  Class  definitions  can  be  modified  simply  by 
changing  an  instance  variable. 


It  should  be  noted  that  the  approach  using  flavor  definitions  provides  a 
means  of  changing  previously  defined  instances  when  a class  definition  is 
changed.  This,  however,  does  not  seem  to  offer  any  significant  advantage 
since  reinitializing  activity  instances  after  changing  class  definition  appears 
to  be  computationally  tractable  and  logically  in  sequence. 
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After  default  activity  templates  and  the  mission  have  been  defined,  sub-act  templates  arc 
generated  during  an  initialization  process.  These  templates  are  used  to  create  the  instances 
of  activities  which  arc  instantiated  throughout  a simulation.  Since  these  templates  may 
provide  functions  to  be  evaluated  at  run-time  for  determining  the  value  of  an  activity 
parameter,  this  approach  provides  a flexible  mechanisms  for  context  dependent  activities. 

Structurally,  any  default  activity  template  can  be  used  to  represent  the  top-level  node  in  an 
activity  hierarchy.  Normally  one  default  activity  template  is  defined  to  represent  a 
procedure  associated  with  a specific  domain  function  (e.g.  "enter  NA V waypoint").  This 
activity  template  will  specify  the  next  lower  level  activities  in  the  activity  hierarchy  with 
lower  level  activities  defined  in  the  default  activity  templates  for  the  indicated  subactivities. 
As  described  in  previous  sections,  terminal  activities  represent  the  bottom  nodes  of  an 
activity  hierarchy  which  is  the  level  at  which  an  agents  actions  interact  with  equipment. 
This  process  is  recursive  until  the  indicated  subactivities  are  terminal  activities.  The 

resultant  activity  hierarchy  represents  the  desired  procedure. 


Figure  2.  Activity  Data  Structures 
5.2.1. 2 Activity  Templates 

AOTV^^Lm<^IJY'TCI^LATES0and  sub-ACT-TEMPLATES  are  defined  with 
. v S as  a comP°nent  flavor,  as  illustrated  in  Figure  2.  ACTIVITY- 
secticmy  mclude  the  following  parameters  (full,  detailed  structure  is  describe  in  later 


act-name 

act-short-name 

agent-allocation 

equip-context 

act-keys 


act-goal-keys 


=>  a string  representing  the  activity  name 
=>  a string  abbreviating  activity  name  - for  graphing 
=>  which  agent  should  perform 
=>  specifies  required  crew  station  design 
=>  provides  a generalized  parameter  - value  list  structure: 
parameter- 1>  <value>  ...<parameter-n>  <value> 

This  enables  new  parameters  to  be  defined  without  recompiling 
=>  used  to  pass  parameters  to  activity  instances. 
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act-type 


required-resources 


functions  => 

priority 

initialization-procedures 
preconditions 
com-fcn-procs 

explicit-local-constraints  = 


estimated-duration 

start-procedures 

tick-procedure 

termination-conditions 

termination-procedures 

vacp-data 

matching-pattern 

sub-act-list 


=>  indicates  general  temporal  relationships  of  subactivities 
"sequential"  specifies  subactivities  should  be  processed  in  sequence, 
"parallel"  specifies  all  subactivities  should  start  when  activity 
starts  assuming  constraints  allow.  Indicates  subactivities  are 

anticipated  to  be  all  started  at  once. 

"rotation"  specifies  sequential  order  which  is  to  be  repeated  until 

termination  conditions  cause  activity  to  stop. 

"complex"  specifies  all  subactivities  should  try  to  start.  Indicates 
constraints  are  anticipated  to  delay  the  start  of  some  actmues^ 
=>  specifies  required-resources,  activities  may  be  constmn<^from 
starting  when  a required  resource  is  not  available.  These 
resources  include: 
visual 
auditory 
cognitive 
motor 

left  / right  hand 
point-of-fixation  (look-at  site) 

^ high  level  text  description  of  activity  for  display  purposes. 

> currently  user  defined  priority. 

> procedures  to  be  run  when  activity  is  initialized. 

> state-activity  pairs  for  indicating  possible  preconditions/acti 

> defined  only  for  terminal  activities.  Specifies  a list  of  equipment 
component  functions  which  to  be  executed  in  sequential  order. 

:>  wiii  be  tested  when  activity  tries  to  start  Indicates 

subactivities  are  anticipated  to  be  delayed  from  starting. 

=>  expressed  in  ticks  - 100  ms.  . 

=>  procedures  to  be  run  when  activity  is  started. 

=>  specifies  forms  to  be  evaluated  each  tick. 

=>  specifies  forms  to  evaluated  each  tick.  When  ?”h®.  . 

forms  returns  "TRUE",  the  activity  will  be  will  be  terminated. 
=>  specifies  forms  to  be  evaluated  when  the  activity  terminates. 

=>  context-free  estimate  of  VACP  values  for  this  activity. 

=>  used  to  match  activities  to  goals. 

=>  used  only  for  decomposable  activities. 


5.2.1. 3 Activity  Instances 

Activity  instances  are  created  during  the  simulation  and  represent  a unique  instance  of  an 
SSSS  Parameter  for  activity  instances  include  the  followtng  (fuU 

structural  details  provided  in  later  sections): 


act-name 

act-id 

agent-allocation 

act-keys 


act-goal-keys 

act-type 


• a string  representing  the  activity  name.  rTAPS'i 

. string  produced  by  ACT<gensym-number>  (e.g.  ACT425). 

. which  agent  should  perform. 

« provides  a generalized  parameter  - value  list  structure. 
<parameter-l>  <value>  ...<parameter-n>  <value> 

This  enables  new  parameters  to  be  defined  without  recompiling 
the  template  definition. 

> used  to  pass  parameters  to  activity  instances. 

> indicates  general  temporal  relationships  of  subactivities 
"sequential"  specifies  subactivities  should  be  processed  in  sequence, 
"parallel”  specifies  all  subactivities  should  start  when  activity 
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required-resources 


priority 

initialization-procedures 

preconditions 

com-fcn-procs 

estimated-duration 

start-procedures 

tick-procedure 

termination-conditions 

termi  na  tion -proced  ure  s 

vacp-data 

mapped-goal 

tick-started 

tick-ended 

sub-activities 

activity-history 


starts  assuming  constraints  allow.  Indicates  subactivities  are 
anticipated  to  be  all  started  at  once, 
rotation"  specifies  sequential  order  which  is  to  be  repeated  until 
termination  conditions  cause  activity  to  stop. 

"complex"  specifies  all  subactivities  should  try  to  start.  Indicates 
constraints  are  anticipated  to  delay  the  start  of  some  activities 
->  specifies  required-resources,  activities  may  be  constrained  from 
starting  when  a required  resource  is  not  available.  These 
resources  include: 
visual 
auditory 
cognitive 
motor 
left-hand 


right-hand 

point-of-fixation  (look-at  site) 

=>  currently  user  defined  priority. 

=>  procedures  to  be  run  when  activity  is  initialized. 

=>  state-activity  pairs  for  indicating  possible  preconditions/actions. 
->  defined  only  for  terminal  activities.  Specifies  a list  of  equipment 
component  functions  which  to  be  executed  in  sequential  order 
=>  expressed  in  ticks  - 100  ms. 

=>  procedures  to  be  run  when  activity  is  started. 

=>  specifies  forms  to  be  evaluated  each  tick. 

=>  specifies  forms  to  evaluated  each  tick.  When  one  of  the 

fomis  returns  "TRUE",  the  activity  will  be  will  be  terminated. 
— > specifies  forms  to  be  evaluated  when  the  activity  terminates. 

=>  context-free  estimate  of  VACP  values  for  this  activity. 

=>  used  only  for  top  level  activities  - an  instance. 

=>  time  (in  ticks)  activity  was  started. 

=>  time  (in  ticks)  activity  ended. 

=>  used  only  for  decomposable  activities  - active  instances. 

=>  used  to  store  terminated  subactivities. 


Mechanisms  have  been  developed  which  provide  for  automatic  generation  of 

"m^POn]etnC  n)°de.  con^mjands-  when  a terminal  sub-act  template  is  instantiated  the 
make-instance  after  methods  tests  each  procedure  in  the  list  of  equipment  component 
function  procedures  and  appropriate  anthropometric  commands  are  inserted. 


EXAMPLE: 


Assume  an  activity  is  defined  with  the  following  component  function 
procedures: 

comp-fcn-procs  =>  ( cfp-1  cfp-2  cfp-3  cfp-4) 

cft-1  representing  a component  function  of  switch  "X” 
cfp'2  representing  a component  function  of  display  "Y" 
cfp-3  representing  a component  function  of  switch  "Z" 
cfp-4  representing  another  component  function  of  switch  "Z" 

Testing  the  first  procedure,  "cfp-1",  the  equipment  model  for  switch  "X"  is 
queried  for  a predefined  site  location  and  a command  is  generated  for  the 
appropriate  "reach-for"  action.  Testing  "cfp-2"  would  generate  a "look-at" 
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command.  It  should  be  noted  that  both  ”cfp-3"  and  cfp-4"  will  generate  the 
same  reach  command,  resulting  in  the  following  list  (code  simplified). 

comp-fcn-procs  =>  ( (reach-for  "X  ) 
cfp-1 

(look-at  "Y") 
cfp-2 

(reach-for  "Z") 
cfp-3 

(reach-for  "Z") 
cfp-4 ) 

It  should  be  noted  that  unless  some  other  activity  is  overlapping,  Ael^t  reach  command 
will  be  asking  for  a hand  to  move  to  its  current  position.  As  detailed  in  the  Jack 
documentation,  Annex  G,  a request  for  a movement  that  is  currently  achieved  wdjbe 
signaled  bv  a completed  message  during  that  tick.  This  results  in  the  mechanism  being 
context  sensitive  in  that  only  movements  that  have  to  be  made  will  have  a duration  beyond 
one  tick  The  one  tick  duration  of  the  null  movements  has  not  seemed  significant  and  this 
mechanism  appears  sufficient  for  modeling  movement  which  is  conditional  upon  the  initial 

state  of  the  required  resources. 

It  should  also  be  noted  that  at  the  end  of  the  activity  one  hand  would  remain  on  switch 
"Z"  If  it  is  desired  for  the  hand  to  return  to  a neutral  position,  it  can  be  handled  in  two 
ways  FtalteuTer  defining  *e  aciviiy  can  specifically  include  the  movement  dunng  the 
initial  specification  as  follows: 

comp-fcn-procs  =>  ( cfp-1  cfp-2  cfp-3  cfp-4  (reach-for  <site-n>) ) 

Second,  the  user  can  define  an  event-response  goal  indicating  that  the  hand  should  return  to 
a predefined  neutral  position  after  a specified  period  of  inactivity. 

5,2. 1.4  Goal  Definition 

In  order  to  provide  goals  which  definitions  can  be  described  and  changed  without 
encountering  the  problems  associated  with  flavor  definitions,  a mechanism  was  developed 
fn  Phase  IV  w aid  the  process  of  goal  definition  that  is  clo^ly  related  to  the  process  used 
for  activity  definition.  As  with  activity  definition,  the  key  feature  is  the  use  of  template  da 
structures  instead  of  flavor  definitions.  Templates  are  used  to  define  default  goal 
parameters  and  specify  global  changes  to  a goal  class  definition,  for  specialization  of  both 
class  and  instances  to  provide  exceptions  to  goal  class  default  values,  and  to  provide 
context  sensitive  activity  instances. 


Templates  are  provided  at  two  level: 

• First,  default  templates  which  provide  a definition  of  a general  goal  class. 

• Second,  subgoal  templates  which  provide  specialization  of  default  goal  templates 
based  on  the  context  of  the  mission  and  the  environment. 


Default  goal  templates  are  defined  by  the  users  during  a pre-simulation  goal  definition 
phase  Subgoal  templates  are  later  automatically  generated  during  a pre-simulation 
initialization  which  must  be  performed  after  the  mission  definition  phase  has  been 
completed. 
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. „ wants.i°  change  a parameter  value  associated  with  all  instances  of  a specific  goal 
timn’i  ? /ecjficatl0ns  l"1the  defau]t  goal  template  should  be  changed  The  default  goal 
templates  for  decomposable  goals  indicate  how  sub-goal  templates  should  be  created8for  its 
subgoals.  If  the  subgoals  are  to  be  created  with  the  values  JS a defaSK^mnSe 

using  me  aeiautt  values,  if  the  user,  however,  wants  to  make  changes  to  the  default 

when  certain  conditions  apply  or  for  a subset  of  a goal  class,  the  Scan  specify  ^ 

conditions  in  the  parent  default  goal  template  that  one  its  subgoals  should  becreated  fWm  n 

H rWlthJ,f  UeS  °f er  i_he  defaults  <see  ^ter  section!  for  details)  iSTprovS 
logic  for  controlling  inheritance  both  globally  and  locally.  ’ P^viaes 

Default  goal  parameters  are  defined  using  DEFAULT-GOAL-TEMPLATES  These 
templates  are  instances  of  the  flavor  DEFAULT-GOAL-TEMPLATEand  die  deS 
parameters  are  specified  as  values  of  instance  variables  These  temnlate  instance  a r 
denn^eflniK0n  £ ^ g°alS-  After  d^uluoal  temjhres 

arJnwl  ^~8°f  t®mPIates  generated  during  an  initialization  process.  These  templates 
Since  the  lnstances  °f  £oals  which  are  instantiated  throughout  a simulation 

Since  these  templates  may  provide  functions  to  be  evaluated  at  run-tune  for  determining  the 
value  of  a goal  parameter,  they  provides  mechanisms  for  contS^^^  ^ 

In  a manner  similar  to  activities,  structurally,  any  default  goal  template  can  be  need  m 

J[En' the  t0P~leve’ node  in  a goal  hierarchy.  Normally  one  default  goal  template  is 
defined  to  represent  the  planned  mission.  This  goal  template  will  soecifv  the  ne*t 

m ,h'  80al  hi'rf  < TOS 
list  smicture  is  compose  of  symbols  and/or  sublists.  The  symbols  in  the  list  refer  to  names 

of  default  goal  templates  which  in  turn  can  specify  a lower  level  of  suhfrnak  th*  c ki* 

m Km,inal  goals- llK  rcsu"an' 8oal 

A,*Pug.h  some  contingencies  can  be  incoiporated  into  the  planned  goals  another 

sssiKa. 

P 8 ^ hierarchies  is  constrained  primarily  by  the  agent's  resources  The 

SKSST  Kqums ' ““ 1 1 “>  deflne  resol“'io"  “i'"  coE’^S,eThe 
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Figure  3.  Goal  Data  Structures 


5.2.1.5  Goal  Templates 

GOAL-TEMPLATE  is  a flavor  that  is  used  as  a flavor  component  of  the  flavor  definitions 
for  both  DEFAULT-GOAL-TEMPLATE  and  SUB-GOAL-TEMPLATE  definitions,  as 
illustrated  in  Figure  3.  The  following  instance  variables  are  common  to  both: 


goal-name 

context-name 

goal-id 

goal-ref-id 

goal-level 

goal-priority 

terminal-p 

goal-keys 

matching-pattern 

matched-act-template 

event-response 

goal-test 

agent-allocation 

agent 

functions 

activation-tests 

currency-tests 

initialization-procedures 

preconditions 

contin-conditions 

sub-goal-list 

tick-procedures 

termination-condi  tions 


=>  a symbol  representing  a goal  name. 

=>  name  with  context  added 
=>  creates  unique  id 

=>  a unique  id  set  by  the  default  goal  templates 
=>  highest  goal  is  level  1 
=>  larger  values  have  priority 
=>  achieve  or  maintain  if  terminal 
=>  misc  keys 

=>  pattern  to  match  goal  to  design  activities 
=>  template  indicated  by  pattern  above 
=>  what  happens  to  template  after  event  is  true 
=>  function  indicating  achievement  status 
=>  who  is  responsible  for  this  goal 
=>  instance  of  agent 

=>  intended  state  changes  achieved  by  activity 
=>  event  tests  for  event-response  goal 
=>  when  is  this  goal  actually  a requirement 
=>  to  be  completed  when  goal  is  instantiated 
=>  list  of  state/goal-templates  pairs  necessary 
=>  list  of  state/goal-templates  pairs  necessary 
=>  list  of  goal  names  and  keywords 
=>  what  to  do  each  tick 
=>  when  to  end 
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termination-procedures 

temporal-relations 


=>  what  to  do  when  I end 

=>  describes  temporal  relations  of  goals 


^Fvar^We^AL^ -TEMPLATES  include  the  above  instance  variables  and  the  following 

s^rce-flle  =>  file  defining  this  goal 

subgoals-func  =>  functions  for  determining  mission  subgoals 


sUB-GOAL-TEMpLATEs  include  the  instance  variable  defined  by  GOAL-TEMPLATE 
and  the  following  two  instance  variables: 

parent-template  =>  goal  templates  creating  this  subgoal 

sub-goals  =>  subgoals 

5.2. 1.6  Goal  Instances 


Goal  instances  are  created  during  the  simulation  and  represent  a unique  instance  of  a task 
requirement.  Parameter  values  are  inherited  from  the  related  goal  template  or  derived  by  an 
function  provided  by  the  template.  Parameters  for  goal  instances  include  the  following  (full 
structural  details  provided  in  later  sections):  6 v 


goal-name 

context-name 

goal-id 

parent-goal 

mission 

goal-level 

goal-priority 

terminal-p 

goal-keys 

act-goal-keys 

matching-pattern 

matched-activity 

event-response 

status 

achievement-status 

template 

agent 

sub-goal-templates 

sub-goals 

activation-tests 

currency-tests 

current-p 

tick-created 

tick-activated 

tick-terminated 

precondit 

contin-condit 

goal-test 

achieved-p 

tick-procedures 


=>  goal  name. 

=>  name  with  context 

=>  creates  unique  id 

=>  higher-level  goal 

=>  instance  of  crew  mission 

=>  highest  goal  is  level  1 

=>  larger  values  have  priority 

=>  nil  when  decomposable,  t if  terminal 

=>  misekeys 

=>  used  to  pass  arguments  to  methods  creating  activities 

=>  pattern  to  match  goal  to  design  activities 

=>  instance  of  activity  result  of  pattern  above 

=>  t if  this  goal  is  an  event  response  goal 

=>  describes  activity  response  to  goal 

=>  specifies  if  goal  is  satisfied 

=>  goal  template  specifying  default  values 

=>  who  is  responsible  for  this  goal 

=>  templates  used  to  create  sub-goals 

=>  instances  of  goals 

=>  event  tests  for  event-response  goal 

=>  when  is  this  goal  actually  a requirement 

=>  tick  when  this  goal  became  current 

=>  tick  goal  created  and  initialized 

=>  when  goal  became  an  active  goal 

=>  when  goal  became  inactive 

=>  list  of  state/goal-templates  pairs  necessary 

=>  list  of  state/goal  - templates  pairs  necessary 

=>  function  indicating  achievement  status 

=>  t or  nil  list  for  maintenance  goals 

=>  what  to  do  each  tick 
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termination-conditions 

termination-procedures 

data-pit 


=>  when  to  end 

=>  what  to  do  when  goal  ends 

=>  used  to  store  data  concerning  state  values 


5. 2.1.7  Goal/Activity  Matching 

A key  feature  of  the  Phase  IV  approach  to  agent  modeling  is  the  mechanism  developed  for 
the  agent  to  select  an  activity  as  an  attempt  to  satisfy  a given  goal.  Since  a primary 
objective  of  the  MIDAS  system  is  to  have  agent  behavior  sensitive  to  changes  in  either 
mission  or  crew  station  design,  a flexible  mechanism  was  neededto  match  each  missmn 
voal  to  a variety  of  activities  associated  with  alternative  crew  station  designs.  This 

Lso  be  sensitive  to  situations  in  which  the  mission  is  nnedand  specific 
SSi.  can  be  used  for  a variety  of  missions.  The  approach  used  m Phase dV 
involves  the  use  of  matching  patterns  to  map  activities  to  goals.  Matching  patterns  are 
defined  for  each  goal  and  this  pattern  is  used  by  a matching  function  to  identified  the 
appropriate  activity.  Matching  patterns  may  be  explicitly  defined  in  the  default  goal 
templates  during  the  definition  phase: 


matching-pattern  =>  '(cpg  send  :FM1  msg) 

They  may  also  be  defined  by  functions  specified  during  the  definition  phase  which  will 
produce  a context  sensitive  pattern  at  run  time  : 

matching-pattern  =>  (,(comm-task-assign  ‘mission*  ’fit-coord)  ;;;  backquote  implied 
send 

,(task-radio-type  ‘mission*  'flt-coord 

(comm-task-assign  ‘mission*  ’flt-coord)) 

msg) 


In  this  example,  the  forms  preceded  by  the  comma  (backquote  implied  incodenot 
shown)  are  evaluated  at  run  time  within  the  context  of  the  nussion  scenario.  The  first 
fom  evaluated,  ".(comm-task-assign  ‘mission*  ’flt-coord)",  accesses  the  crew- 
mission  object  by  the  global  variable  ‘mission*  and  checks  to  see  which  agent  is 
currently  asssigned  flight  coordination  ("flt-coord")  responsibilities.  The  second  form 
evaluated:  ” (task-radio-type  ‘mission*  'flt-coord  (comm-task-assign  mission  flt- 
coord))",  identifies  which  radio  should  be  used  for  this  task.  The  resulting  pattern 

could  be  one  of  the  following: 

matching-pattem  =>  '(pilot  send  :FM1  msg) 
matching-pattern  =>  ’(pilot  send  :VHF  msg) 
matching-pattem  =>  '(pilot  send  :UHF  msg) 
matching-pattem  =>  '(cpg  send  :FM1  msg) 
matching-pattem  =>  '(cpg  send  :VHF  msg) 
matching-pattem  =>  ’(cpg  send  :UHF  msg) 

These  matching  patterns  are  then  passed  as  an  argument  to  a agent’s  matching  method, 
(MAP-ACTIV1TY-TO-GOAL  AGENT)  which  returns  the  appropnate  activity  instance.  In 
Phase  IV  simple  hashing  was  used  in  the  matching  method  since  it  serve  the  intended 
S Of  SJnsS  how  the  mapping  could  be  context  sensitive.  For  more  complex 
problems,  a more  powerful  mapping  function  may  be  desired  such  as  one  using  a many 
sorted  logic.  Newmapping  techniques  may  be  easily  integrated  simply  by  redefining  the 
agent's  "MAP-ACTIVITY-TO-GOAL"  method. 

An  imoortant  feature  of  using  the  matching-pattem  approach  is  that  goals  and  activities  can 
be  defined  in  a modular  manner  and  links  between  goals  and  activities  defined  later  an  wi 
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dlow  separate  and  parallel  development.  This  is  important  since  it  became  clear  during 
Phase  IV  that  a symbolic  modeling  approach  to  simulation  is  extremely  labor  intensive  and 
will  require  a significant  team  to  work  on  all  but  the  most  trivial  design  models. 

5.2.1.8  Integrating  Jack 

An  anthropometric  model  developed  at  the  University  of  Pennsylvania,  commonly  referred 
to  as  Jack,  was  integrated  with  the  Symbolic  Operator  Model  in  Phase  IV.  Jack  was  used 
to  model  body  movements  and  drive  graphical  displays  during  the  simulation.  By 
incorporating  Fitt's  Law  into  the  reach  movements,  Jack  was  able  to  provide  a model-based 
estimate  of  the  duration  required  to  perform  various  reaches.  Head  movements  were  also 
provided  for  the  simulation;  however,  these  movements  were  either  made  by  having  the 
head  follow  die  hand  during  a movement  or  by  simple  shift  of  position  which  would  be 
accomplished  in  one  tick  (100  ms).  Integration  of  Jack  was  accomplished  by  predefining  a 
set  of  three  dimensional  sites  and  having  terminal  activides  generating  commands  to  reach 
(or  look)  at  a given  site. 

These  commands  were  implemented  in  the  form  of  component  functions  and  could  be 
intermixed  with  component  functions  defined  by  equipment.  The  duration  of  these 
commands  would  be  determined  by  the  Fitt's  Law  estimate  provided  by  Jack.  The 
commands  generated  a data  set  which  was  passed  to  a data  pool  (see  Annex  J). 

The  data  set  generated  by  these  commands  have  the  form: 

(<data-pool-identifier>  <reach-type>  <reach-site>) 


<data-pool-identifier> 

<reach-type> 

<reach-site> 


=>  an  integer 
=>  an  integer 

=>  a string.  The  first  character  of  the  string  is  significant 
and  limited  to  the  following  rules: 
ii^n  identifies  the  site  is  on  the  right  multifunction  display 
L identifies  the  site  is  on  the  left  multifunction  display 
"x"  is  used  for  all  other  sites 

command  is  generated  to  reach  to  a new  site,  the  reach  type  deteimines  if  the  reach 
is  made  from  the  shoulder  or  the  waist  by  checking  the  reach  type  code  which  also 
determines  if  the  head  will  follow  the  hand  during  the  movement.  After  the  initial  tick  the 
£°!  Wllj  “Pd,ate  the  appropriate  variables  for  the  agent  including  a variable  for  the 
A , y , and  Z position  of  the  hand  and  the  status  variable.  The  status  variable 
indicates  the  remaining  time  to  finish  the  movement  and  will  indicate  "0M  when  the 
movement  has  been  completed.  Agent  objects  can  check  to  see  if  a hand  resource  is 
committed  simply  by  checking  the  the  status  variable  for  that  hand. 

It  should  be  noted  that  many  movements  are  affected  by  the  last  movement  performed  For 
example,  during  testing  a command  was  generated  which  cause  the  CPG  agent  to  reach 
down  and  to  the  rear  of  where  the  hand  had  been  located.  This  cause  the  anthropometric 
model  to  generate  a bending  of  the  wrist  which  seemed  appropriate  for  that  movement 
However,  the  next  command  generated  a reach  to  a site  on  the  multi-function  display. 
During  the  execution  of  this  movement,  the  wrist  was  not  straightened  and,  since  the  reach 
was  for  the  dp  of  the  finger  to  touch  the  screen,  most  of  the  hand  was  "pushed”  through  the 
display  so  the  finger  could  make  contact.  There  were  also  some  movements  of  the  head  to 
the  rear  that  should  have  been  possible  but  appear  illegal  since  the  head  would  fall  off 
uunng  Phase  IV,  these  problems  were  interesting  but  not  serious  problems  since  realistic 
movements  could  be  generated  after  some  trial  and  error.  A more  serious  issue  arose  from 
the  requirement  to  predefine  3-D  sites.  The  number  of  sites  in  a typical  crew  station  could 
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be  extremely  large  and  the  requirement  to 

g££££SSd  Sfp?obUms  if U was  dILd  to  have  the  agent  tok  at  amoving 
3£S*  (S«  "“jack  documentation.  Annex  G,  for  a complete  dtscusston  of  the 
anthropometric  model.) 

5,2. 1.9  Integrating  the  Task  Loading  Model 

One  intended  use  of  a MID  ***£££ Icc 

SSS£“S£^3SSa 

where  overloads  would  have  been  experienced. 

sssi =»-£S2SSSSS 

the  TLM  models  concerning  requirements  in  the  definition  pnas  . 
requirements,  see  the  Task  Loading  Model  documentation.  Annex  C. 

5.2.1.10  Integrating  the  Scheduler 

S5-S5E3E—SSSS? 

strategy  may  prove  w”S  loped  to  provide  additional 

simulation  by  allowing  the  sdtedute to 

»S523K3^^E^™  SSSh— 
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associated  with  multiple  goals  in  a manner  to  minimize  time  or  to  maYimiTf  use  0f 
resources. 

Although  the  integration  of  the  scheduler  was  not  shown  during  the  demo,  a simple 

vaTus  times  durin8  development  This  mechanism  is  dearly 

hl  Jina  ^ bUl’  Slt!Ze  mechanism  be  replaced  with  more  effidcnt  modules  without 
having  to  make  other  changes,  it  provides  means  of  evaluating  the  use  of  this  type  of 
scheduler  m a simulation. 

B2eP'io"  approach  used  during  Phase  IV  development  was  to  modify  the  agent's  tick 
method  in  the  following  manner.  Normally,  after  the  activities  are  sorted  by  goalpriority 
resources  are  allocated  to  each  activity  in  sequence  unless  that  activity  is  restricted  from 
starting  by  a constraint.  "Hie  modified  agent's  tick  method  added  a new  step  after  the 
activities  were  sorted  and  before  the  activities  were  allocated  resources.  This  new  step 
wrote  acuvity  details,  including  constraint  descriptions,  to  a file  which  was  read  by  the  Z- 

Thi* ?h5i'  iAt  th,SJX)i.nt 016  agenJ  model  waits  until  a new  ffle is  written  by  the  Scheduler 
Th^  ?fih?  i rea,ds  th?1fcQ?',,y  description  file  and  evaluates  it  as  a scheduling  problem 
The  solution  developed  by  the  Scheduler  is  expressed  as  a set  of  additional  constraints 
which  are  written  to  the  file  that  the  agent  model  is  waiting  to  read.  Upon  reading  the  new 
constraints  from  the  file  written  by  the  Scheduler,  the  agent  proceeds  in  the  normal  manner 
of  activating  activities  and  allocating  resources  as  permitted  by  constraints  which  now 
includes  the  new  constraints.  Although  the  method  of  passing  data  by  writing  to  a file  is 

c ear  y simplistic  and  inefficient,  it  can  be  replaced  by  a more  sophisticated  method  without 
requiring  changing  other  mechanisms.  “ wunout 

It  should  be  noted  that  the  Z-Scheduler  used  in  Phase  IV  takes  a series  of  tasks  and 
constraints  as  input  and  returns  a amended  constraint  list  as  its  output.  Additional 

SE"***8  W1  ?eed  lo1bc,deJvf.IoPed  t0  address  situations  in  which  the  ordering/reordering 
, , ?DJl  iets  result  in  task  shedding.  Also,  since  the  scheduler  requires  estimates  of  each 
tasks  duration  which  is  independent  of  task  sequencing,  the  scheduler  is  not  sensitive  to 

o/preomditionsf  h°W  1116  °f  Vari°US  3Ctivity  schedules  316  effected  by  reach  times 

5.2.1.11  Output  Available  for  Analysis 

During  a MIDAS  simulation,  each  terminated  goal  and  activity  is  stored  for  later  analysis 
goafob)e0cVt  S " °r  Perforraance  since  S°al  satisfaction  is  stored  within  Lh 


^!TeS  pr<?^ld®in{ormatlon  required  to  analyze  how  crew  station  design  affects  crew 
activity  hfs1oriCfLUIeS CU,TCn"y  rcC°"i'd  for  analysis  b“‘ ca" 


5.2.2  Detailed  Design  Description 

Details  concerning  the  design  of  the  basic  objects  are  provided  in  the  following  sections. 


5.2.2.1  Scenarios 

Objects  of  flavor  SIM-SCENARIO  have  the  following  instance  variable  values: 
SIM-NAME:  User  defined  string  "Enroute  Segment  3". 

SIM-ID:  Not  currently  used. 
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ENVIR-CONTEXT: 


WORLD-OBJECTS: 


Used  to  record  and  access  context  dependent  information. 

This  variable  is  a list  with  the  following  format: 
( <parameter-l>  <parameter- 1 -value> 
<parameter-2>  <parameter-2-value> 

<parameter-n>  <parameter-n-value>) 

Example: 

(:PHASE-LINES 

((RED  48008400  48007900  D-1/26C  A V) 

(BLUE  50008400  50007900  B-1/26CAV)) 

ACPS 

((ACP-DELTA  48208030  TOC)) 

:FARP  47008000 

:FLOT  '(54008400  54007900)) 

Objects  which  should  be  given  a tick  message. 


E-TRANSMISSIONS:  Instance  of  radio  messages. 

INITIAL-TIME:  Time  tick  0 represents  (e.g.  tick  0 is  0630  - 6:30  AM). 


CURRENT-TIME:  Time  in  military  format. 


CURRENT-TICK:  Time  in  ticks. 


CONTROL-EVENTS:  Not  currently  used.  Included  to  provide  a structure  for 

adding  events  controlled  by  the  simulation.  The  structure 

is  of  the  form: 

( (<test-l>  <event-l>) 

(<test-2>  <event-2>) 


(<test-n>  <event-n>) 

Each  form  is  tested  every  tick:  if  the  test  evaluates  to  true,  the 
event  is  executed  (e.g.  a message  being  sent). 


INT-CUED-E VENTS:  Not  currently  used. 

INT- ACTIVE-EVENTS:  Not  currently  used. 


INT-TERMINATED-EVENTS:  Not  currently  used. 


5. 2. 2. 2 Crew-Missions 

Objects  of  flavor  CREW-MISSION  have  the  following  instance  variable: 


MISSION-NAME: 

MISSION-ID: 

ASSIGNED-CREW: 

CREW: 


User  defined  string  (e.g.  "Enroute-1"). 

Not  currently  used  - provided  for  managing  multiple  missions. 
List  of  agent  instances. 

Instance  of  the  crew  flavor. 
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MISSION-ROLE: 

Symbol  designating  role  (e.g.  TM1-AC1  =>  Aircraft  1 in  Team  1) 
This  symbol  is  also  referenced  in  the  CEOI. 

CALLSIGN: 

Standard  military  format: 

<alpha><numeric><alpha><numericxnumeric> 
Y5T46  =>  "Yankee  Five  Tango  Four  Six” 

EQUIP-SYSTEMS: 

Not  currently  used. 

GOAL-NET: 

Symbol  used  to  reference  top  level  activity  used  to  represent 
the  planned  mission  (e.g.  ENROUTE-1). 

WAYPOINTS: 

Hash  table  of  waypoints. 

POINT-OF- DEPARTURE:  Grid  coordinates  of  take-off  point  (e.g.  43407910). 

ROUTES: 

List  of  predefined  routes  in  the  following  format: 

( (<route-number>  ( <waypoint-a>  ...<waypoint-n>)) 
(<route-number>  ( <waypoint-a>  ...<waypoint-n>)) 

(<route-number>  ( <waypoint-a>  ...<waypoint-n>)) ) 
Example: 

((1  (1  2 3 4))  (2(1  4))  (3(12  3567))) 

DEFAULT-ROUTE- A/S : 

Not  currently  used  - but  provided  for  interface  purposes. 

DEFAULT-ROUTE- ALT:  Not  currently  used  - but  provided  for  interface  purposes. 

PLANNED-ROUTE: 

List  of  planned  route  in  the  following  format: 

( (<route-number> 

( <waypoint-a>  <altitude-to-wp>  <airspeed-to-wp>)) 
( <waypoint-b>  <altitude-to-wp>  <airspeed-to-wp>)) 

ASSIGNED-ROUTE: 

( <waypoint-a>  <altitude-to-wp>  <airspeed-to-wp>)) 
Example: 

(1  (1  50  50) 

(2  50  50) 

(3  50  50) 

(4  50  0)) 

List  of  currently  assigned  route.  This  will  be  the  same  as  the 
planned-route  unless  the  route  changes  during  the  mission. 

LAST-PLANNED-ROUTE:  For  debugging  purposes. 

WAYPOINTS-FLOWN: 

List  of  waypoint  id  integers  for  waypoints  overflown. 

NEXT-WP: 

Should  be  deleted  - use  next-waypoint  instead. 

NEXT-WAYPOINT: 

Integer  representing  id  of  next  waypoint. 

CURRENT-ROUTE-HDG:  Not  currently  used  - provided  for  future  display  purposes. 

CE0I:  Provides  information  relating  to  communication  tasks. 

Represents  information  normally  provided  by  the 
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Communications  Electronic  Operating  Instructions  (CEOI). 

Uses  the  following  format; 

^parameter- 1>  parameter- 1 -value> 

<parameter-2>  <parameter-2-value> 

<parameter-n>  <parameter-n-value>  ) 

During  Phase  IV  the  format  was  specialized  to: 

( xallsigns  (<entity-l>  <callsign-l> 

<entity-2>  <callsign-2> 

<entity-n>  <callsign-n>)  , , N 

met  ( (<comm-net-l  <net-radio-type>  <net-fireq>  <stnng-for-display>) 


Example  from  Phase  IV  demo; 

(:CALLSIGNS 

((BN-CDR  P7F78) 
(CO-CDR  Y5S92) 
(TOCY5M14) 
(TM1-LD  Y5T46) 
(TM1-AC2  Y5T52) 
(TM1-AC3  Y5T73) 
(TM2-LD  Y5V43) 
(TM2-AC2  Y5V67) 
(TM2-AC3  Y5V25) 
(FARP  Y5D37) 
(D-1/26CAV  G3W54) 
(B-1/26CAV  H6L39) 
(ADA  J2Q87)) 


Battalion  Commander 
Company  Commander 
Tactical  Operations  Center 
Lead  - Team  1 
Aircraft  2 - Team  1 
Aircraft  3 - Team  2 
Lead -Team 2 
Aircraft  2 - Team  2 
Aircraft  3 - Team  2 
Forward  Ann  Refuel  Point 
D Troop,  1/26  Cavalry 
B Troop,  1/26  Cavalry 
Air  Defense  Net  Control 


:NET  ;^KnY:FM1  38.5 (CO-CDR TM1-LD TM2-LD) 

(COMPANY°VHF l5?  (CCWDR  TM2-LD  TM2-AC2  TM2-AC2 ) 

(TEAMI-C^SSf^ 234HcicDR  TM1-LD  TM1-AC2  TM1-AC2) 
"Team  1 UHF  command  net") 

(TEAM1  ;FM1  40.9  (TM1-LD  TM1-AC2  TM1-AC3) 

"Team  1 internal  FM  net") 

(TEAM1  ;VHF  140.7  (TM1-LD  TM1-AC2  TM1-AC3) 

"Team  1 internal  VHF  net") 

(TEAM2  :FM1  44.0  (TM2-LD  TM2-AC2  TM2-AC3) 

"Team  2 internal  FM  net") 

(TEAM2-CMD  :VHF  147.9  (CO-CDR  TM2-LD  TM2-AC2  TM2-AC3) 
"Team  1 VHF  command  net") 

(TEAM2  :UHF  237.2  (TM2-LD  TM2-AC2  TM2-AC3) 

Team  1 internal  UHF  net") 

(TOC  :FM1  54.9  (TOC  BN-CDR  CO-CDR) 

"Bn  command  net") 

(ADA  :FM1  48.5  (ADA  TM1-AC2  TM2-AC2) 

"ADA  net") 

(D-1/26CAV  ;FM1  34.8  (D-1/26CAV  TM1-AC2  TM2-AC2) 
"D-1/26CAV  command  net") 

(B-1/26CAV  :FM1  46.4  (D-1/26CAVTM1-AC2TM2-AC2) 
"B-1/26CAV  command  net"))) 
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DEFAULT-COMM-ASSIGN:  Describes  how  communication  tasks  were  initially  allocated 

In  the  Phase  IV  demo.  The  pilot  was  assigned  to  monitor 
the  command  net  on  UHF  and  the  team  net  on  VHF.  The 
CPG  was  assigned  to  monitor  the  company  FM  net  and 
make  required  flight  coordination  calls  on  FM  which 
would  require  changing  frequency  between  monitoring 
company  FM  and  making  required  flight  coordination  calls. 

Example  from  Phase  IV  demo: 

(rPILOT 

((TEAM1-CMD  :UHF) 

(TEAM1  :VHF)) 

:CPG 

((COMPANY  :FM1) 

(FLT-COORD  :FM1))) 

COMM-ASSIGN:  pis  should  be  different  from  default-comm-assign  only  if  there  has 
been  a reallocation  of  tasks  during  the  mission  (i.e.  function 
allocation  for  load  leveling,  etc.) 


COMM-REQ. 

List  of  symbols  indicating  a requirement.  This  provides  an  easy 
means  of  editing  and  changing  an  agent's  requirements  of  editing  and 
changing  a crew  task  requirements. 

Example: 

COMM-REQ:  (REPORT-CROSSING-PHASE-LINES) 

LAST-POS-REPORT:  Time  (in  ticks)  of  last  position  report.  This  is  hard-coded  as 

instance  variable  only  to  demonstrate  capabilities.  In  the  future, 
this  should  probably  be  one  of  many  parameters  in  a more 
generalized  variable. 

5.2.2.3  Agents 

An  object  of  flavor  AGENT  has  the  following  instance  variables: 

ID: 

Automatically  generated  id  string  in  the  format  of 
Agent-<a  gensymed  number>  (e.g.  "Agent-355"). 

CREW: 

Not  currently  used  - intended  for  agent  abstraction. 
Agent  abstraction  specifies  that  one  agent  of  the  crew 
should  be  allocated  this  goal  but  does  not  indicate  if  it 
is  the  pilot  or  cpg. 

ROLE: 

Role  of  agent  - a symbol  - Pilot  or  CPG 

;;;  pe  following  instance  variables  are  specific  to  the  anthropometric  model. 
’.!!  d 6 t“”ocument^on  f°r  this  model  for  details  concerning  the  values. 

;;;  Keach/Head-tum  ids  specify  how  the  model  should  move  (e.g.  from  the 
,„  waist  or  shoulder,  etc).  Sites  specify  a predefined  3-d  location  to  which  a 
;;;  movement/look  should  be  made. 

LH-REACH-ID: 

LH-REACH-SITE: 

LH-X: 

LH-Y: 
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LH-Z: 

LH-STATUS: 

LH-LAST-SITE: 

RH-REACH-ID: 

RH-REACH-SITE: 

RH-X: 

RH-Y: 

RH-Z: 

RH-STATUS: 

RH-LAST-SITE: 

HEAD-TURN-ID: 

SITE-TO-LOOK-AT: 

HEAD-YAW: 

HEAD-PITCH: 

LAST-SITE-LOOKED-AT: 

;;;  end  of  anthropometric  variables 


E-R-GOALS: 

GOALS: 

TERMINAL-GOALS: 


Instances  of  event-response  goals  not  yet  activated. 
Instances  of  activated  decomposable  goals. 
Instances  of  activated  terminal  goals 


GOAL-FILE: 


Instances  of  terminated  goals 


...  The  following  variables  were  included  to  keep  track  of  where  fe  agent  had 
!!!  Kgen  looking  These  values  could  be  tested  to  see  if  the  agent  had  been 
:::  SSJ Squired  items.  This  was  not  fully  developed  in  the  demo  and 
probably  should  be  removed  as  hardcoded  variables  and  developed  within 
;;;  the  representation  of  a cognitive  architecture. 

LAST-CHECKED- ALT: 

LAST-CHECKED- A/S: 

LAST-CHECKED-HDG : 

LAST-CHECKED-PWR: 

L AST-CHECKED-TRAN : 

LAST-CHECKED-FREQ: 

L A ST-CHECKED-C  W 1 : 

LAST-CHECKED-LWIN: 

LAST-CHECKED-RW1N : 

LAST-CHECKED-FWIN: 

LA  ST-CHECKED-N  A V : 

;;;  end  of  variable  group 


ACTIVITIES: 

CUED- ACTIVITIES: 
TERMINAL-ACTS: 


Decomposable  activity  instances 

Activities  not  yet  started 

Terminal  activity  instances  currently  executing 


ACT-FILE:  List  of  terminated  activities 

SCHEDULER-TRIGGER:  Not  currently  used. 
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VISUAL-REF: 

INT-TEMP-VACP: 

RESOURCE-COMMIT: 

VACP-HISTORY: 

ASSERTIONS: 

MSGS-REC: 


Not  currently  used. 

Used  to  allocate  resources  to  activities. 

Used  to  record  how  resources  are  allocated. 

Not  currently  used  - had  been  simple  recording  of  time-tagged 
additive  VACM  values. 

Not  currently  used  - intended  for  conclusions  based  on  rule 
interpretation  of  perceptual  data  fusion. 

Not  demonstrated  in  demo  - but  provided  to  handle  the 
processing  of  messages  monitored. 


An  object  of  the  flavor  agent  has  the  following  methods: 


Methods  for  accessing  and  setting  the  all  local  instance  variables. 
MAP- ACTI V ITY-TO-GO AL  -see  code  for  documentation 
TICK  - see  code  for  documentation 


HEAD-MOVE-COMPLETED-P 

RH-REACH-COMPLETED-P 

LH-REACH-COMPLETED-P 

LOOK-AT 

RH-REACH 

RH-BLIND-REACH 

RH-REACH-FROM- WAIST 

RH-REACH* 

LH-REACH 

LH-BLIND-REACH 

LH-REACH-FROM-WAIST 

LH-REACH* 


5.2.2. 4 Goals 
Objects  of  flavor  GOAL  have  the 
GOAL-NAME: 
CONTEXT-NAME: 

GOAL-ID: 

PARENT-GOAL: 


following  instance  variables: 

User  defined  string  (e.g.  "FLY-TO-WP"). 

User  defined  format  statement  which  produces 
a context  sensitive  string  (e.g.  "FLY-TO-WP2"). 

Automatically  generated  id  string  in  the  format  of 
G-<a  gensymed  number>  (e.g.  "G-355"). 

Instance  of  a goal  which  specifies  this  goal  as  a subgoal. 
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MISSION: 

GOAL-LEVEL: 

GOAL-PRIORITY: 

TERMINAL-P: 

GOAL-KEYS: 


ACT-GOAL-KEYS: 

MATCHING-PATTERN: 


MATCHED- ACTIVITY : 
EVENT-RESPONSE: 

STATUS: 

ACHIEVEMENT-STATUS : 
TEMPLATE: 

AGENT: 


Instance  of  crew-mission. 

Integer  - higher  numbers  represent  lower  levels  in  a hierarchy. 

Integer  - higher  numbers  represent  higher  priority. 

T or  NIL  - T if  terminal  goal,  NIL  otherwise. 

Used  to  record  and  access  context  dependent  information. 
This  variable  is  a list  with  the  following  format: 

( parameter- 1>  <parameter-l-value> 

<parameter-2>  <parameter-2-value> 

* • • * 

<parameter-n>  <parameter-n-value>) 

Only  used  by  terminal  goals. 

Used  to  pass  parameters  to  activities. 


Only  used  by  terminal  goals. 

A pattern  that  is  defined  by  the  user  and  is  passed  to 
a matching  function  to  identified  an  act^ty 
may  satisfy  the  goal  (e.g.  FLY-COURSE-TO  WP) 


Only  used  by  terminal  goals. 

Not  currently  used  - flag  now  indicated  by  having  a form 
in  die  activation-tests  variable. 


Not  currendy  used. 

Not  currendy  used  - achieved-p  used  instead. 


For  debugging  purposes. 

Instance  of  agent  - the  goal  was  allocated  to  this  agent. 


SUB-GOAL-TEMPLATES : 


SUB-GOALS: 

ACTIVATION-TESTS: 


Only  used  by  decomposable  goals.  ......  . 

Used  to  store  references  to  subgoals  which  should  not 
be  created  when  the  goal  is  initially  created  (e.g.  al 
but  the  first  subgoal  for  a sequential  goal)  but  wdl  be 
created  later. 

Only  used  by  decomposable  goals.  , 

List  of  goal  instances  which  are  subgoals  of  this  goal. 

Only  used  by  event-response  goals. 

Specifies  a form  to  be  evaluated  each  tick.  When  the 
form  evaluates  to  true,  the  goal  is  activated.  Event- 
response  goals  are  stored  in  the  event-response  goal 
instance  variable  (i.e.  e-r-goals)  of  the  agent  and  the 
activation  test  of  each  goal  in  this  list  is  tested  each 
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CURRENCY-TESTS: 


CURRENT-P: 

TICK-CREATED: 

TICK-ACTIVATED: 

TICK-TERMINATED: 

PRECONDIT: 

CONTIN-CONDIT: 

GOAL-TEST: 


tick  by  the  agent's  tick  method.  When  the  goal  is 
activated,  it  is  removed  from  the  list  and  placed  in 
the  appropriate  decomposable  or  terminal  goal  list. 

Used  to  test  satisfaction  erf  the  goal.  Defined  by  the  user. 
The  use  of  backquotes  is  common  here  since  the 
user  can  specify  when  a subform  should  be  evaluated 
During  development,  the  use  of  embedded  forms 
using  backquotes  was  tested  so  evaluation  of  user 
defined  forms  is  performed  in  two  stages.  The  first 
stage,  completed  during  the  initialization  phase,  results 
in  the  specialization  of  the  embedded,  second  form 
which  is  evaluated  during  the  simulation. 

For  example,  the  goal  for  sending  a required  message 
has  a currency  tests  variable.  During  the 
definition  phase,  the  user  defines  the  goal  with  the 
following  form  (the  backquote  is  not  shown): 

:currency-tests  (.Gist  '>  '(utm-x  ♦aircraft*) 

(-  (cadr  report)  20))) 

After  initialization,  the  currency-test  has  been  specialized 
to  the  following  by  evaluating  the  backquoted  form 
which  produces  a form  which  can  test  the  if  the 
x-coordinate  location  of  the  aircraft  is  greater  than 
20  meters  before  the  phase  line  : 

:currency-tests  (>  (utm-x  *aircraft*)  4780) 

The  form,  (>  (utm-x  *aircraft*)  4780),  is  then  tested 
during  the  simulation. 

Indicates  if  the  goal  is  current  - T or  nil. 

Time  (in  ticks)  goal  was  created. 

Time  (in  ticks)  goal  was  activated. 

Time  (in  ticks)  goal  was  terminated. 

Not  currently  used. 

Not  currently  used. 

Appropriate  only  for  terminal  goals. 

Specifies  how  goal  satisfaction  should  be 
tested.  A list  with  the  following  structure: 

( <goal-temporal-type>  <goal-test-form>) 

Goal  temporal  type  can  be  either  ACHIEVE  or 
MAINTAIN.  The  goal  test  form  can  be  any  valid 
lisp  form  which  returns  T or  nil. 

Example: 

(MAINTAIN 

(EQL  (COURSE-IND  ♦AIRCRAFT*)  76)) 
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ACHIEVED-P: 


Indicates  if  goal  has  been  satisfied. 

Achievement  goals  are  classified  as  either 
satisfied  or  not-satisfied. 

Maintenance  goals  specify  states  (e.g.  mamtein 
heading  135”)  which  must  be  maintained  for 
a specified  duration.  Maintenance  goals  are 

cuirentiy^satSied  = state  maintained  but  goal  has 

not  yet  been  completed 

not-currently-satisified  = state  not  maintained  goal  has 

not  yet  been  completed 

satisfied  = state  maintained  and  goal 

completed 

not-satisfied  = state  not  maintained  and 

duration  completed. 


TICK-PROCEDURES : 
TERMINATION-CONDITIONS: 


Not  typically  needed  but  retained  for  special  purposes. 


Tested  each  tick  - if  evaluates  to  True  the  goal  is 
terminated. 

ET<  (DISTANCE-TO  * AIRCRAFT*  10170  3445)  400)) 


TERMINATION-PROCEDURES: 


rocedures  which  should  be  run  when  goal  terminates. 

* _ii. . Ki-it  T-Atoin fnr  snecial  DUTDOSCS. 


DATA-PIT: 


Used  to  store  terminated  subgoals. 


5. 2.2.5  Activities 

SSESSSSSSSSSBr 

the  following: 


act-name 

context-name 

act-short-name 

act-id 


a string  representing  the  activity  name. 

a string  including  context  dependent  descriptor 

a string  abbreviating  activity  name  - for  graphing. 

string  produced  by  ACT<gensym-number>  (e.g.  ACT425). 


agent-allocation 

act-template 

equip-context 

act-keys 


vhich  agent  should  perform, 
lsed  for  debugging, 
ipecifies  crew  station  design. 

provides  a generalized  parameter  - value  list  structure: 
<narameter-l>  <value>  ...<parameter-n>  <value> 

This  enables  new  parameters  to  be  defined  without  recompiling 
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the  template  definition. 


act-goal-keys  used  to  pass  parameters  to  activity  instances. 


act-type 


required-resources 


functions 

priority 


indicates  general  temporal  relationships  of  subactivities 
sequential  specifies  subactivities  should  be  processed  in  sequence 
parallel  specifies  all  subactivities  should  start  when  activity 
starts  assuming  constraints  allow.  Indicates  subactivities  are 
anticipated  to  be  all  started  at  once, 
rotation"  specifies  sequential  order  which  is  to  be  repeated  until 
M termination  conditions  cause  activity  to  stop. 

"complex"  specifies  all  subactivities  should  try  to  start.  Indicates 
constraints  are  anticipated  to  delay  the  start  of  some  activities. 


specifies  required-resources,  activities  may  be  constrained  from 
starting  when  a required  resource  is  not  available.  These 
resources  include: 
visual 
auditory 
cognitive 
motor 
left-hand 
right-hand 

point-of-fixation  (look-at  site) 


high  level  text  description  of  activity  for  display  purposes, 
currently  user  defined  priority. 


initialization-procedures 

preconditions 

com-fcn-procs 


procedures  to  be  run  when  activity  is  initialized. 

state-activity  pairs  for  indicating  possible  preconditions/actions. 

defined  only  for  terminal  activities.  Specifies  a list  of  equipment 
component  functions  which  to  be  executed  in  sequential  order. 


remaining-comp-fcn-procs  used  internally  to  keep  track  of  which  component  function  has 

been  completed.  Initially  this  list  is  set  to  a copy  of 

comp-fcn-procs.  As  component  functions  are  completed,  they 
are  deleted  from  this  list.  y 


explicit-local-constraints  will  be  tested  when  activity  tries  to  start.  Indicates 

subactivities  are  anticipated  to  be  delayed  from  starting. 

estimated-duration  expressed  in  ticks  - 100  ms. 


start-procedures 

cf-procedures 

tick-procedure 

termination-conditions 


procedures  to  be  run  when  activity  is  started. 

not  currently  used  - use  com-fcn-procs  instead. 

specifies  forms  to  be  evaluated  each  tick. 

specifies  forms  to  evaluated  each  tick.  When  one  of  the 

forms  returns  TRUE  , the  activity  will  be  will  be  terminated. 
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termination-procedures 
compute- vac  p-data 

vacp-load-form 

vacp-data 

vacp-data-hi  story 

mapped-goal 

tick-cued 

tick-started 

tick-ended 

sub-activities 

activity-history 


specifics  forms  to  be  evaluated  when  the  activity  terminates. 

flag  indicating  VACP  values  should  be  determined  by  form  in 
the  vacp-load-form  slot 

form  used  to  produce  context  dependent  VACP  values 

context-free  estimate  of  VACP  values  for  this  activity. 

time-tagged  VACP  values. 

used  only  for  top  level  activities  - an  instance. 

time  (in  ticks)  activity  was  initially  tested  for  starting. 

time  (in  ticks)  activity  was  started. 

time  (in  ticks)  activity  ended. 

used  only  for  decomposable  activities  - active  instances, 
used  to  store  terminated  subactivities. 
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6.0  USER'S  GUIDE 

6.1  INSTALLATION  AND  INITIALIZATION 


Files  for  the  Symbolic  Operator  Model  are  maintained  in  a directory  called  MIDAS2  This 

directory  is  organized  into  subdirectories  as  follows: 


BASIC1 

BASIC2 

COMM 

DEMO 


- Defines  flavor  definitions  and  global  variables. 

- Defines  basic  functions  and  methods. 

- Defines  data  pool  as  required  by  the  communications  module. 

* ?ef“?es  mission,  objects,  agents,  goals  and  activities  specific  to  the 
the  Phase  IV  demonstration. 


DISPLAYS  - Defines  the  Symbolic  Operator  Model  display  constraint  frame. 


The  Symbolic  Operator  Model  has  been  defined  as  a system  using  the  Genera  Develooment 
System  Tools  in  a file  named  "midas.system"  in  the  MIDAS2  directory.  ^ 

Loading  the  system  requires  that  the  Equipment  Modeling  and  LONGBOW  systems  be 
The  Ph^e  IV  demonstration  also  requires  a bin^y fite  SSSm 
from  the  MIDAS  directory.  This  file  defines  a terrain  digital  elevation  model  (DEM)  which 
was  used  dunng  development.  Since  it  was  the  stated  objective  of  the  Program  Office  to 
maintain  DEM's  only  on  the  IRIS  machines,  the  code  should  be  changed^fSess 

S5BBSSSS the  ms  instead  of  from  the  structurc  011681641  b* the  binaiy  fdc  ™ 

Mmlco  S.ymb°uiC  9Pfrator  Model  at  a new  site>  the  newest  lisp  files  from  the 
M^HSridU^r  ^ sh°uld  be  copied  to  1116  new  site-  Besides  the  Symbolic  Operator 
Modd,  files  defining  the  Equipment  Models  and  the  Longbow  System  will  be  reauired 
Details  for  installing  these  systems  are  provided  in  Annex  D.  Files  required  for^ 
communications  are  detailed  in  Annex  J and  these  files  must  be  loaded  prior  to  any 
,As?*.  l^e  ?urrent  Symbolic  Operator  Model  requires  one  binaiy  file  y 

' fr0m  the  ^1IDAS  directoty-  This fileshould  be recreated iind  moved 
IDAS  directory  into  the  MIDAS2  directory  for  future  development  (the  MIDAS 
directory  was  used  as  the  development  directory).  P MiUAJ) 

Although  it  is  not  required  prior  to  loading  the  system,  prior  to  running  any  simulation  it  k 

fi,eHS  by  lhe  ^onmu n i ca ti on s'mr^ulesTndinMaHre 

the  IRIS  machines  as  instructed  in  the  Communications  module  documentation.  Annex  J. 

After  the  required  files  have  been  loaded  onto  the  fileserver,  the  file  "midas  system”  must 

be  edited  to  reflect  any  changes  in  logical  pathnames.  s.sys  must 

Programmers  should  be  aware  that  many  of  the  files  in  the  "DEMO"  directory  were  created 

deveWnir  d.emo.nstration  in  IV.  Any  future  experimentation  with  code 
developed  will  require  changes  to  many  of  the  definitions  provided  in  these  files 

Experimentation  with  Jack  can  be  accomplished  without  changing  files  in  other  directories. 
The  Symbolic  Operator  Model  may  then  be  loaded  by  the  following  top  level  commands: 
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prompt>  Load  System  Midas2 


The  system  will  load  the  Equipment  Modeling  and  Longbow  systems  if  they  are  t 
tam  SKEwt  environment.  Any  files  required  by  the  Commumcattons  module 
th«n  he  loaded. 


not  loaded 
should 


The  Symbolic  Operator  Model  display  may  be  selected  by: 


SELECT  Symbol-Shift-E 


Initialisation  is  accomplished  by  selecting  the  "RESET  S>M''menuten in dte  MIDAS 
display.  The  system  is  ready  for  a simulanon  after  this  has  been  completed. 


6.2  STARTUP  AND  TERMINATION 

Simulation  starting  and  termination  is  accomplished  by  Simulation  Executive  commands. 
Refer  to  Annex  J for  details. 
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MAN-MACHINE  INTEGRATION  DESIGN  & ANALYSIS  SYSTEM 
(MWAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 

PHASE  IV: 


Z - SCHEDULER 


1.0  INTRODUCTION 


1.1  IDENTIFICATION  OF  DOCUMENT 


This  document  establishes  the  requirements  and  detailed  design  of  t 

of  the  MIDAS  system.  The  Z-Scheduler  document  has  to  be  read  within  the  broader  context 
of  the  documenSn  of  the  Symbolic  Operator  Model  (Annex  A)  and  the  Task  fading 
Model  (Annex  C). 


1.2  SCOPE  OF  DOCUMENT 

The  material  in  this  document  is  directed  towards  three  categories  of  readers  as  shown  m 
Figure  1 below. 
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The  three  categories  of  readers  are: 

UfrfrhcdBlcr  Users:  Those  who  are  interested  in  learning  what  Z-Scheduler  Module  in 
MIDAS  does.  The  users  interact  with  the  Z-scheduler’s  user  interface  layer  primarily  to 
input  a set  of  tasks  and  view  the  schedule  through  the  user  interface.  No  programming  skills 
are  required  to  interact  with  the  scheduler  in  this  capacity. 

2)  Z-Sghgdlllcr  Applications  Programmers:  Those  who  wish  to  extend  the  capabilities  of  the 
Z- Scheduler  software.  For  example,  when  one  wants  to  build  upon  or  modify  the  existing 
set  of  scheduling  strategies  in  the  scheduler.  Some  lisp  programming  and  basic  GEST 
programming  knowledge  is  recommended  for  using  Z-Scheduler  for  this  purpose. 

3)  Z-Scheduler  tool  programmer:  Those  who  might  want  to  modify  and  update  the  Z- 
Scheduler  tool  layer.  Extensive  knowledge  of  the  Symbolics  programming  environment, 
object-oriented  programming,  knowledge-based  designing,  and  some  experience  in 
programming  with  GEST  is  required  to  program  Z-Scheduler  in  this  capacity. 

1.3  PURPOSE  AND  OBJECTIVE  OF  DOCUMENT 

Thi^°^r?ent  attempts  to  describe  the  methodologies  used  to  represent  the  Z-Scheduler  of 
the  MIDAS  system.  The  Z-Scheduler  was  proposed,  designed  and  implemented  in  Phase  IV 
of  the  A31  project.  This  implies  that  though  there  has  been  some  references  to  the 
Scheduling  model  in  the  previous  phase  documentation  there  has  never  been  a powerful  and 
separate  scheduling  model  in  the  earlier  phases  of  this  project 

2.0  RELATED  DOCUMENTS 

2.1  APPLICABLE  DOCUMENTS 

ThCrfollowing  documents  are  referenced  herein  and  are  directly  applicable  to  this 

Symbolics  Genera  7.2  Documentation,  Symbolics  Publication  Number  999079 
Symbolics,  Inc.,  Cambridge,  Massachusetts,  1988. 

GEST  ver4.0  manual,  GTRI,  Atlanta,  GA,  1989. 

2.2  INFORMATION  DOCUMENTS 

The  following  documents  amplify  or  clarify  the  information  presented  in  this  volume: 

J/™e.s  Alien  "Maintaining  Knowledge  about  Temporal  Intervals",  Communications  of  the 
ALM  26  (11),  832-843,  1983. 

David  Ben-Arieh,  Knowledge  Based  Control  System  for  Automated  Production  and 
Assembly  , Purdue  Univ,  Ph.D  Dissertation,  Aug  85. 

T£j5arch  paperS  and  Publications  1981-1987,"  NASA  Technical  memorandum 

lUUUlb,  170/. 

Barbara  Hayes-Roth,  "A  BlackBoard  Architecture  for  Control"  , A1  26  (1985)  251-321. 
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Barbara  Hayes-Roth  and  Frederick  Hayes-Roth.  "A  Cognitive  Model  of  Planning", 

Cognitive  Science  3,  275-310  (1979). 

N P Keng  D Y Y.  Yun  and  M.  Rossi,  "Interaction  Sensitive  Planning  System  for  Job- 
Siop  Scheduling".  In  Expert  Systems  and  Intelligent  Manufacturing,  Michael  D.Oliff 
editor.  Elsevier  Science  Publishing  Co.,  1988. 

Johan  de  Kleer,  "An  Assumption  Based  TMS”,  AI  28  (1985)  127-162. 

Jay  Liebowitz,  Patricia  LightFoot,  "Expert  Scheduling  Systems:  Survey  and  Preliminary 
Design  Concepts",  Applied  A I 1:261-283, 1987. 

N Morray,  M.  Dessouky,  "Scheduling  Models  for  Worldoad  Prediction  and  Guidance", 
EPRL  report,  U.  of  Illinois  at  Urbana-Champaign,  Aug  89. 

Jeff  Rickel  "Issues  in  the  Design  of  Scheduling  Systems".  In  and 

Michael  D.Oliff  editor.  Elsevier  Science  Publishing  Co.,  1988. 

P.  Sanderson,  N.  MotTay,  'The  Human  Factors  of  scheduling  Behavior",  EPRL  report  # 
90-09,  U.  of  Illinois  at  Urbana-Champaign. 

Peng  Si  Ow,  Stephen  Smith,  and  Randy  Howie,  " ^ 

Expert  Systems  and  Intelligent  Manufacturing,  Michael  D.Oliff  editor.  Elsevier  Scie 
Publishing  Co.,  1988. 

Peng  Si  Ow,  Stephen  Smith  , "Viewing  Scheduling  as  an  Opportumstic  ^oblem-solvmg 
Process",  Annals  of  OR.  Approaches  to  Intelligent  Decision  Support  , R.  G.  Jeroslow 
(editor),  Baltzer  Scientific  Publishing  Co.,  1987. 

3.0  CONCEPT 

3.1  DEFINITION  OF  Z-SCHEDULER 

7 Scheduler  is  a constraint-based,  opportunistic  scheduling  tool  that  represents  one  of 
SSJs? fiX 'SStiSSoS.  The  Z-Scheduler  is  provided  with  a task  queue  along  with 
52?at»ut  each  fade (such  as  temporal  constraints,  estimated  duration,  resource 
iauiSents  etc)  ^ scheduler  solves  for  a "near-optimal"  sequence  and  schedule  based 
on  a sttategy  of  time  minimization  or  load  balancing,  intended  to  reP^nt  ^«ble  operator 
behaviors  Z-Scheduler  uses  a blackboard  architecture  — the  closest  thin^o  a cognmve 

architecture  to  date.  Also,  this  architecture  allows  its  components  to  be  modular  and  easily 
^ teSle  The  components  of  Z-Scheduler  are  called  knowledge  sources  and  each 
com^'^pSSa  s,agc  in  .he  scheduling  process.  Z-Scheduler^  f^  m^ulnr^.s 
call^  on  an  "as-needed"  basis  by  the  task  decomposmon  with  a variable  temporal  honzon. 
Z-KSukr  closely  interacts  wtih  the  Task  Loading  Model  for  learning  about  resource 

fe£S3S35Si£55fiB3* 

local  solutions  along  to  derive  the  global  scheduling  solution. 

3.1.1  Purpose  and  Scope 

The  orimarv  purpose  of  the  Z-Scheduler  is  to  extend  the  power  and  applicability  of  the 
MIDAS  symbolic^ operator  model.  As  mentioned  above,  Z-Scheduler  works  at  modeling  the 

pilot's  strategic 
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scheduling  behavior.  Specifically  the  Z-Scheduler  takes  a set  of  tasks  generated  bv  the 
planner  as  input  and  outputs  a specific  order  on  the  execution  of  the  tasks.  Z-Scheduler  has 
bee".  keePing  strict  modular  principles  in  mind  and  hence  the  scope  of  Z-Scheduler's 
applicability  is  therefore  wide.  C i 

3.1.2  Goals  and  Objectives 

The  primary  goal  of  the  Scheduler  was  to  capture  the  scheduling  behavior  of  an  operator 
uito  a computational  model.  The  focus  of  the  work  has  been  modeling  the  scheduling 
sffategies  that  the  operator  applies  while  scheduling  behaviors  to  achieve  desired  results 

n?mS,fth!dU'er  1S- Currendy  geared  ‘ovulate  two  such  operator  scheduling  strategies 
namely,  the  mimmize-time  strategy  and  the  balance-load  strategy.  The  former  strategy  is 

h^^-°h1S  c?ncern^  Primarily  with  performing  all  the  allocated  tasks  within  the 
P™  window  by  extending  his/her  resources  to  the  maximum  limit  possible.  The 
strategy  simulates  the  operator  delaying  the  execution  of  the  tasks  so  as  to 
continue  being  at  his/her  comfortable  resource  load  limit. 

In  the  earlier  phases  of  this  project,  the  pilot  activities  whose  preconditions  are  satisfied  at 
any  moment  in  time,  begin  executing  at  that  moment.  There  was  minimal  and  in  some  cases 
no  mechanism  to  do  the  resource  allocation  in  order  to  arrive  at  an  opportune  moment  to 
perform  the  task.  In  other  words,  there  was  no  "look  ahead"  mechafisrn by wKhe 
operator  anaiyzes  the  currently  active  activities  and  decides  on  an  order  for  performing  the 
task  such  that  the  tasks  all  get  done  in  a more  efficient  manner.  This  is  the  aspect  of  the 

th^7 Zhrtt  0131  thC  Zfchedu'er  was  tarSeted  to  address.  Hence,  a more  general  goal  of 
the  Z-Scheduler  was  to  enhance  the  symbolic  operator  model.  ^ 

Four  major  objectives  have  been  outlined  for  the  Z-Scheduler  for  its  first  developmental 
stage  during  Phase  IV  of  the  A3l  project.  They  are  the  following: 

1.  Simulate  varied  operator  scheduling  strategies. 

2.  Reflect  changes  in  schedule  with  a change  in  environment. 

3.  Provide  for  predictive  task  analysis. 

4.  Assist  in  enhancing  the  operator  model  with  a look-ahead  feature. 

IS*?**  objective  of  the  Z-Scheduler  was  that  Z-Scheduler  should  be  able  to  demonstrate 
scheduling  behavior  obtained  as  a result  of  applying  varied  scheduling  strategies  It  has  been 

deoenden/on  t T ^ °n  ^havior  that  a hunfaS Sule  is 

S , h heuristics  applied  during  scheduling  and  these  heuristics  are  what  gives 

nse  to  strategies.  In  this  phase  of  development  of  the  scheduler,  we  isolated  two  such 
proven  strategies.  We  then,  scheduled  tasks  using  the  two  strategies  as  objective  functions 

«^”f«gS'dU'eSandi 

r!n  ?r°adeSt  8ial  of  MIDASf  “ t0  show  the  imPact  of  the  man-machine  interface 
design  on  human  performance,  it  follows  that  all  modules  of  MIDAS  obey  this  guiding 
principle.  Since,  Z-Scheduler  assigns  a time-line  order  to  a set  of  tasks  performed  by 
humans  while  interacting  with  the  man-machine  system  design  (in  Phase  IV  the  man- 
machine  system  was  the  Apache  helicopter),  it  can  be  easily  seen  that  if  the  environment 
changes  the  set  of  tasks  will  change  and  so  will  the  schedule.  Hence,  using  this  logic  we  can 
conclude  that  Z-Scheduler  reflects  changes  in  the  schedule  with  a change  in  environment. 
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Hie  ms*  input  ,o  me  scheduler  ere  me  us*  ^ •K3ffl.‘tfaSK55& 
hSJflSSKf  Mid “mat” me  mne  window  that  me  scheduler  is  working  on  fttttng  all 


horizon) — . , . 

the  tasks  is  in  some  projected  future  tune 


By  incorporating  a sc 

our  Phase  IV  objectives  ts  ^ss^suchas.^  ^ ^ ^ ^ (o  ^ ^ 
d tasks? 

c!  What  happens  wherTthe  '^ot  ch'angesliiVtmh^uling  strategy  (like  optimizing 


scheduler  in  the  MIDAS  symbolic  operator  model,  questions  pertinent  to 


a.  How  close  to  an  < 

re'alisTffi?.  on  me  schedule,  when  the <* SST" 


time,  or 


■> 

Display  (MFD)  environment? 

Z-Scheduler  is  designed  to  handle  the  above  queries;  hence  justifying  me  requirement  for  a 
scheduler  in  the  MIDAS  symbolic  operator  model. 

3.1.3  Description 

Z-Scheduler  aitempts  to  capture  varied  operaicu  scl^ulitig  into  a^com|^ocm^ 

model.  Two  such  strategies  have  be  P haiance  ioaci  strategy  During  the  application 

Scheduler  - the  minimize  nme  strategy  ^d  me  ^ , 

of  me  minimize  time  sffategy 1 * PPf  , resoun;e  capabilities  to  me  maximum  level 

The  input  to  Z-Scheduler  is  a set  of  tasks  Kk 

planning  component  of  *«««* t S^SSTof  a task  to  the 

ssssr^t^s^^  * - - — *• 

the  Task  Loading  Model. 

Along  with  the  stack  of  tasks,  Z-Scheduler 

as  some  restriction  placed  on  the  Th  auajitativc  constraints  in  the  literature  are 

two  tasks.  For  example,  we  on  ^ ,aSk  «n  be  dLtly  t^hotd  to  me 

relationships  defined  in  (J.  Allen,  1983).  A > •ctart.a»  »ask-A  5 secs".  The  effort  of 

into  rx 

is  om  of  the  most  significant  contributions  of  the  Z-Scheduler. 

mmmmsm 


Page  B-5 


ESssasBta®  ;:Se  sr'  »*M£ERR2i  We 


"™™™S 10  the  he2? °[,the, scheduling  process,  the  blackboard  architecture  is  central  to 
-scheduling  process.  The  blackboard  is  a central  structure  that  contains  all  the  scheduling 
sute  data  and  is  divided  into  sections,  each  section 

generation  process.  The  knowledge  required  to  perform  the  schedule  is  also  narri tinned  nn  j 
each  partition,  called  a knowledge  source,  works  on  each  stage  of  the  scheduling  process. 

source  to  be  triggered  is  the  constraint  solver  knowledge  source  The 
ffus  if  *ta*  knowledge  source  is  to  translate  the  hybrid  represenmdonXonsd^its 
into  a uniform  framework  for  the  temporal  reasoning  process  Also  because  of  the  nature  nf 
inputs  to  Z-scheduler  the  tasks  are  oblivious  of  the SeSSm  HenST  com^nT 
solver  contains  rules  for  solving  for  one  constraint  at  a time  that  includes  caching  the 

dependency  ^p^"'  ***  *’*  incrementally  bui]din2  networks  called  paitiaf 

t“k'based  scheduling  for  partitioning  the  scheduling  problem  SDace 
Taskbasedscheduling  is  a two  step  process.  The  first  step  involves  the  selection  of  a single 
task  from  the  task  stack  that  contains  only  the  unscheduled  tasks  The  second  sten  intnlvS 
thecon^tment  of  a resource  time  slice  to  the  selected  task  byevaluftinT^ 

ScTispxs1  - 

source  and  lh‘  second  oui 

Thf™6?0  *n  fdccbng  ^5ks  for  ^source  allocation  is  called  the  task  criticality  measure 
J”6  h»s  three  factors,  one  of  them  being  the  slack  on  that  taskX  WdTslack 
the  task  less  its  task  criticality.  The  heuristic  applied  in  the  task  selector  knowledge 
scHmce  « that  the  task  with  the  least  amount  of  slack  becomes  the  candidate  for  resource 
allocation  The  metric  for  allocating  a resource  time  slice  for  the  scStSskL 
solution  that  has  minimum  resource  competition,  the  logic  being  that  the  solution  must  not 
™ZT thC  opportun.,t>' of  ‘he  tasks  that  have  not  yet  been  scheduled.  This  metric  is 
kSSg^!rw.C°mPC,lUOn  mCaSUre  and  ‘S  imP]emented  hy  the  resource  allocator 

The  constraint  propagator  knowledge  source  is  a record  keeping  knowledge  source  Fverv 

™Ca  ,S  tCheduledi‘  Upda-tes  rest  of  the  task  windows  accordingly  by  pSSK 

temporal  constraints  m the  partial  dependency  graphs.  P P S S 

?tivK  VVher^ver  3 high  ievel  monitor  of  the  constraint  propagating 

Whenevcr  a conflict  occurs  during  the  process  of  constraint  propagating  this  8 

knowledge  source  halts  the  execution  of  the  other  knowledge  sources  by  Sne  a message 
via  the  blackboard.  After  halting  the  process  that  was  responsible  for  the  conflict  fan  ^ 
example  of  the  conflict  is  that  the  window  duration  of  the  task  becomes  smaller  than  th+ 
dura, ion  of  Ac  ask)  d*  mnh  main, nine,  m!emp“ 
conflic,  s occunence  and,  on  finding  i,  on.,  rase, s all  the  knowledge  sources  ,o  ftejoim  in 
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time  that  brought  about  the  conflict.  In  other  words,  the  truth  maintainer  is  built  on  the 
principle  of  chronological  backtracking,  where  the  amount  backtracking  is  the  previous 
decision  point. 

3.2  USER  DEFINITION 

The  Z- Scheduler  resides  within  the  symbolic  operator  model  of  the  MIDAS  system  and 

between  the  Task  Generator  and  the  Simulation  module  as  shown 

in  Figure  2 below: 

Input  from  other  modules  Output/Simualtion  results 


\ 

Task 

Generator 

Simulation 

Executive 


A3l’s  Pilot  Model  In  Phase  3 


A3l's  Pilot  Model  in  Phase  4 

Figure  2.  Scheduler  in  Relation  to  Symbolic  Operator  Model 
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It  is  important  to  distinguish  Z-Scheduler  from  the  task  generator  and  die  simulation 
executive  components  of  MIDAS.  The  responsibility  of  determining  what  activities  need  to 
be  performed  to  accomplish  the  generated/specified  goals  remains  in  the  task  generator. 
Similarly,  the  functions  of  tick  passing  and  synchronization  with  the  other  models  will 
remain  within  the  simulation  executive. 

As  mentioned  above,  Z-Scheduler  only  determines  the  optimum  order  and  times  for  a list  of 
tasks,  dictated  by  some  objective  function  (such  as  time  or  load  minimization)  which 
represents  the  pilot  s strategic  control  of  his/her  behavior.  To  perform  this  function,  the  Z- 
Scheduler  receives  as  input  a set  of  tasks  with  some  specified  constraints.  The  scheduler 
uses  these  constraints  to  search  through  the  solution  spaces  of  all  feasible  schedules  to  arrive 
at  the  optimal  dme-line  order  of  the  tasks. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

Z-Scheduler  is  a constraint-based,  opportunistic  scheduling  tool  that  attempts  to  capture 
varied  operator  scheduling  strategies  into  a computational  model.  Provided  with  a task 
queue  along  with  data  about  each  task  (such  as  temporal  constraints,  estimated  duration 
resource  requirements,  etc),  the  Z-Scheduler  solves  for  a ''near-optimal"  sequence  based  on 
the  specified  operator  scheduling  strategies.  In  Phase  IV,  we  have  implemented  two  such 
operator  scheduling  strategies  and  they  are  the  minimize  time  strategy  and  the  balance  load 
s!ra!f  ®y-  The  minimize  time  strategy  is  one  in  which  the  operator  works  towards  performing 
all  the  specified  tasks  in  as  short  a time  window  as  possible.  The  balance  load  strategy  is 
one  in  which  the  operator  works  towards  being  at  a comfortable  resource  level  and  hence 
delays  the  execution  of  some  of  the  tasks. 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 


The  operational  scenario  for  demonstrating  the  capabilities  of  the  Z-Scheduler  during  Phase 
IV  was  tasks  extracted  from  the  preflight  segment  of  the  AH64  task  analysis.  This  segment 
included  two  functions  — one  being  the  communications  function  and  the  other  being  the 
auxiliary  power  unit  (APU)  stating  function.  The  former  function  included  two  tasks 
transrmt-cockpit-comniunicarion  and  1 receive-cockpit-communication.  The  APU  starting 
function i included  seven  tasks,  the  higher  priority  tasks  being  set-gen  1 -switch  and 
set-gen-2-switch.  The  rest  of  the  five  tasks  were  routine  monitoring  tasks  the  details  of 
which  are  in  file  puf:>Renuka>Z>tests>apu-starting-tasks.dryrun" . 

Some  of  these  tasks  were  related  to  each  other  by  the  specifications  of  four  constraints.  The 
primary  constraints  were  that  the  set-gen-2-switch  must  be  after  the  set-gen- 1 -switch  and 
that  transmuting  communications  comes  before  receiving  communications.  For  obtaining  the 
details  of  the  other  constraints,  refer  to  the  above  mentioned  file. 

On  starting  the  execution  of  the  blackboard  controller  (for  the  exact  commands,  please  refer 
to  AppendixA),  the  first  stage  of  the  scheduling  process,  the  constraint  solving,  is 
performed.  The  user  will  notice  all  the  input  tasks  stacked  as  cards  in  the  window  titled 
Unscheduled  Task  Stack".  Similarly  the  constraints  specified  by  the  user  will  be  shown  in 
the  Unsolved  Constraint  Stack  window.  As  the  constraint  solver  solves  the  constraints 
incrementally,  the  user  will  notice  constraints  being  pulled  out  of  the  constraint  stack  and  the 
respective  partial  dependency  graphs  (PDG)  being  drawn.  When  the  constraints  are  all 
solved,  the  constraint  solver  redraws  the  original  constraint  stack,  but  this  time  under  the 
heading  of  Applied  constraint  stack".  After  the  constraint  solver  completes  its  function  of 
creating  the  PDGs,  the  constraint  propagator  knowledge  source  knowledge  fires  up. 
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At  this  point  the  users  au'f  0^  of 

ssstA  “teJsr-A  *. 

constraint  propagation  and  works  on  calcu  8.^  fa  he  stack  window.  The 
user  will  notice  the  tasks  being  shuffled  andreorga  t : htcd  for  a small 

tasks  that  have  changed  their  relative  posit  becomes  the  candidate  for  resource 

instant.  The  task  that  gets  to  be  Qf  execution,.*® 

allocation.  When  the  resource  alloca  8 , GANTT  chart  and  a sliding  bar  shift 

user  will  notice  the  selected  task  V*  hgd£«ed  ‘J^gnlfies  the  estimated  _ 

across  the  span  of  the  task  window  ^ ^ s indicates  possible  resource  allocation 

duration  of  the  selected  task  and  the  d p , etes  assigningthe  resources  when  the 

positions  for  that  task  The  from  a double-headed  arrow 

S a"sSa“ed*tox.  TOeausTwiL»  notice  the  cumulative  resource  curves  showtng  up  in 
the  four  resource  panes. 

The  process  then  iterates  between  the  notice  the  A 

propagator  knowledge  source  and, .as  ^scheduled  the  stack  becomes  empty.  At 

" i^SSSSSiS 2ffi!£35*d  task  all  am  marked  by  soltd 

boxes.  * 

For  the  sake  of  clarifying  a display  that  task  across 

color  coded.  Each  task  gets  assigned  a the  user  is 

the  many  windows.  Also,  at  any 'point  £ formed  by  suspending  the 

encouraged  to  inspect  the  tasks  and  constrain^  cs£ ^ performed 

knowledge  source  e*ec“uo" : rf IJSIri.dSe  process WhUe  choosing  the  latter  option  the  user 

maybe  viewing  dati.  dial  migh.  have  already  changed 

since  the  information  was  displayed. 

4.0  REQUIREMENTS 

Z-scheduler  requires  a blackboard  inorder  to  save  onP 

the  blackboard  framework alternatives  we  choose  Generic  Expert  System 

time  and  effort.  Among,  the  few  . tw0  alternative  in  terms  of  cost  to  features 

Tool  (GEST),  since  it  outperformed  the  Jtetw dtemativi si  on  zem.cost  basis 

ratios.  GEST  was  cost  effective,  indeed  its  run  time  veremn  was^  ^ ^ ^ 

to  the  group  and  it  offered  a host  of  fea  ure  ^ . During  the  time  of  evaluation  of 

sseKSsassatea--- 

machines.  Since,  Z-Scheduler  is  a part  of  the  operator  mod,.-  ^ since  GESX-s  favored 
*flymWic"  machmw!  we* decided  dm.  Z-Schedulcr  being  built  on  top  of 
GEST  should  be  on  the  Symbolics  machine. 


4 1 HARDWARE  ENVIRONMENT 
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The  details  of  the  machine  configuration  is  shown  below.  The  details  include  the  circuit 
Chassis'"  Syn,b<>UCS  3640  (hos,  nan,e  is  '•’“to'ttd  is  in  the  AW, 
Datapath 
Sequencer 
Memory  Control 
Front  End 
2Meg  Memory 
CAD  buffer 
512K  Memory 
10 
FEP 

10  Paddle  Card 

K *> MBy SdT>ry  and  27177K  WOrdS  Swwins sp,ce'  ’’“SSdSSa' *? 
The  Ethernet  Address  of  Puffer  is  : 08-00-05-03-20-20 

4.2  SOFTWARE  ENVIRONMENT 

meir  visions  Urrendy  2 WOrld  ,0ad  «“■«”*  Mowing  systems  along  with 

5y$t£rp  Version 

.2 


Genera 
System 
Utilities 
Server  Utilities 
Hardcopy 
Zmail 
LMFS 
Tape 
Nsage 

Documentation  Database 
IP-TCP 

Experimental  News 
Generic  Expert  System  Tool 
Color 

Color  Support 
Color  Doc 
SGD  Book  Design 


7. 

376.158  (ECO  level  6) 
27.29  (ECO  level  4) 
28.5  (ECO  level  1) 
118.17  (ECO  level  1) 
165.20  (ECO  level  1) 
102.7  (ECO  level  1) 
82.18  (ECO  level  3) 
27.227  (ECO  level  1) 
62.1 

67.8  (ECO  level  3) 

5.0 

296.0 

405.13 

409.13 

408.0 
2.6 


GesUoad. r" ** ab°Ve combination  of environment  is  " FEP0:>Inc-Ocean-Color- 
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Hie  interface  between  connection  using  the 

SES^&P&Ji “ SS  %*d>Z  model  is  achieved  by  having  one  image  of 
the  model  resident  within  Z-Scheduler. 

Z-Scheduler  has  also  been  successfully  ported  to  Genera  8.0. 

4.3  EXTERNAL  INTERFACE  REQUIREMENTS 

The  task  generator  CTO  project  into  a ne^  future  sei 

tasks  likely  to  happen  and  tSipori  Constraints  that  might  exist  between  the 

of  tasks  it  also  generates  a set  of  possible > tempo  spawned  by  the  Task  Generator 

tasks  and  appends  this  to  a global  constra  • duradoi{7amount  of  resources  required 
contains  some  task  characteristics  su  ^ 0f  constraints  to  Z.  Also,  the  Task 

StnSoT^" 

Z-Scheduler  generates  all  the  ^ge  ordenng  of  the  assignments  for 

into  Partial  Dependency  Graph ^roGUv^s^i  me  po^  on  ^ ^ 

each  task  by  performing  a ^k-based  P°*  he  Z-Scheduler  closely  interact 

resource  assignment  for  «ch  mk,  t hus » torn  g ®e  interactions  between  tasks  that  may  be 
with  Task  Loading  Modelto  learn  a^fui  Scheduler  is  a iist  of  additional  constraints  that 
performed  concurrently.  The  & ^ rf  ^ ^ 

appends  to  the  global  constraint  1 s , . z The  output  constraints  are  generated 

constraints. 


4.3.1  Interface  with  Task  Generator 

4.3.1.1  Input  from  the  Task  Generator  to  Z-Scheduler 

The  Mowing  are  global  vahables  .hat  are  ahamd  berween  ihe  usk  generator  and  ihe  Z 
Scheduler 

*tasks-to-be-scheduled*  ::=  instances  of  tasks  to  be  scheduled 


♦constraints 


:=  list  of  constraints  on  the  tasks.  Each 
straint  is  a list  with  unspecified  number 
:lements.  The  structure  of  the  list  is 
ined  as  follows.  The  first  element  is  the 
;eory  of  the  disjunction  in  the  constraints. 

-h  of  the  other  elements  consist  of  three  sud- 
ments  - the  first  is  a type  of  constraint 
ond  is  the  task  that  the  constraint  belongs 
ind  the  third  is  the  constraining  element- 
1 be  a task, a constant  time  and  also  a form 
_ 


♦primary-strategy*  ::=  Minimize-time  or  level-load, 
♦time-available*  time  allowed  for  the  tasks  to  be  executed. 
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secondary-strategies*  a list  of  secondary  strategies. 

4.3.1.2  Output  to  Task  Generator  from  Z-Scheduler 

*output-constraint-list*  ::=  list 

; Similar  in  syntax  to  *input-constraint-list* 

, Subset  of  the  disjunction  constraints  in  the 
; input-constraint-list*  plus  all  the  valid 
; singular  constraints  plus  all  the  new  singular 
, constraints  generated  by  Z-Scheduler.  * 

the;6 fol  1 o wingUway  ^ the  mpUt  constraints  list to  create  the  output  constraint  list  in 

L Shind^  nUmbCr  of  choices  in  the  disjunctions  - if  possible  just  leave  only  one 
2.  Leave  the  singular  constraints  undisturbed 

■ or  tasks  to,  have  no  constrain*,  add  constrain*  relating  the  ask  to  other  asks. 
4.4  REQUIREMENTS  SPECIFICATION 

4.4.1  Process  and  Data  Requirements 

S iS  *'  scheduler  is  a se,  of 

SSea!^ 

Scheduler  also  has  mechSto  “ctSvm 10  GEST  *“«•  Z‘ 
constraint  frames.  The  following  section  will  ^ lnp,ut  co.nstraint  into 

requirements  for  the  Z-Schedulfr  as  it  is  repIesemed  jn  GEST  deSCnption  of 

4.4. 1.1  Input  Representation 

Onan  abstract  ievel  Z-Scheduler  is  a system  whose  input  is  described  as  (I.  R,  p,  s,  C) 

R- is  LeftdiSd)FOR  RAD,°'  TUNE'rADIO). 

TUNE-R^DIO)Stra,ntS  0n  R ^ , (F°r  "amPle- ' REACH-FOR-RADIO  is-before 


(I) 


4.4.1. 1.1  Input  Task  Representation 

that  tas^lready^en^rove^  thc  tas^ formalism 

PlanCTate,  Drummond"  89).  °‘ 

others  too  use  the  same  idea  as  in  O-PLAN  i e a fr  “v1  as  um>  .SIS» ISA  and  many 
basic  idea  present  in  these  systems  of  renm^miL  ?PPcmf on  for  the  tasks.  The 

the  Z-Scheduler.  The  task  fi2Sfa£2SBJg  SdSSS  b a,S° PreSent  in 

frames  or  is  set  during  the  manipulations  of  the  fcheVuli^dedS^  ^ t0 
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Task 

name 

agent 

priority 

estimated-duration 

cognitive-resource 

EST-value 

LET-value 

ST-value 

ET-value 

EST-unscheduled-filters 

EST-scheduled-filters 

LET-unscheduled-filters 

LET-sched  uled-filters 
Scheduling  status 
member-PDG 


The  estimated  duration  on  the  task. 

The  VACP  values  on  this  task.  This  slot 
will  interface  to  Task  Loading  Model. 
Earliest  Start  Time  value 
Latest  End  Time  value 
; Start  Time  value 
; End  Time  value 


; Indicates 
; Indicates 


uhpthpr  the  task  has  been  scheduled  or  not. 
which  Partial  Dependency  Graph  the  task  belongs  to. 


4.4. 1.1.2  Resource  Representation  (R) 

The  set  of  resources  (R)  to  rfS  KsourcBtoMC  present  in  our 

S'S  “ ^ fOT  "S 

process. 


Resources 

list-of-all-cognitive-resources 


instantaneous-cumulative-visual-load 

instantaneous-cumulative-auditory-load 


tognitive  resources  are  resources  that 
lay  be  shared.  This  set  is  dictated  by 
'ask  Loading  Model  Hence  if  Task 
wading  Model's  model  has  4(VAC  p) 
hannels,  then  this  will  be  a list  of  4 
lements.  In  the  event  Task  Loading 
dodel  decides  to  have  a 5 
:hannel  model,  then,  it  will  be 
effected  in  the  slot  This  slot  helps 
n the  integration. 

Cognitive  resources  are  analog  in 
nature.  Hence,  small  chunks  of  such 
resources  can  be  allocated  to  separate 

sks  that  are  performed  in  parallel. 

; the  total  visual  resources  allocated 
; at  each  instant. 

same  as  above,  but  for  the  auditory 


instantaneous-cumulative-cognitive-load 

instantaneous-cumulative-motor-load 


4.4. 1.1.3  Primary  Strategy  Representation  (P) 

The  primary  strategy  (P)  of  the  scheduler ^depends .on The * ** 
pilot  wants  to  complete  all  tasks  as  soon  as  pos s . ® scheduling  process  always  applies 

the^rim^ "^raregy1^1^  dedsioV^'m^in^scheduling  cycle.  Hence,  the  primary 
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3%  to  £££53?  °f  •»"-»»  bc«—  *e  aJ  .ornate  scfcdu.es. 


and  in 


; MINIMIZE-TIME  etc. 


Primary-strategy 
strategy 

4.4. 1.1.4  Secondary  Strategy  Representation  (S) 

meas^eTht^oSnS  ?f  SfeSa? SS&  As^ume^S  T*™™ that  m «PPHed  to 
that  equally  satisfy  the  primary  strategy  In  such  StStiSS  rtf™  “*  T*  altemate  schedules 
measures  are  applied  to  select  the  more  optimal  sch^nuT’  * secondary  performance 
■ no.  represenled  cxplici™?„Xde  inn  £ ?S2Jft  'W°'  A"ho"*h  <“• 
shown  below  will  be  implememed  during  fnture  ex°enS  ’ “* S<mcaK  “ 

r»  _ i 


Secondary-strategy 
list-of-strategies 


Each  element  is  a list  that  has  2 elements, 
ine  rirst  is  an  evaluation  function  and  the 
second  is  the  desirable  value  for  it.  For  example 
the  evaluation  function  may  be  ” * 

TOTAL-MOVEMENT-TIME-BETWEEN-TASKS  - sum  r\f 

movement  time  of  all  tasks  from  itself  to  its  **  “ °f 

SlMAL.taSk’  a'ld  thC  dCSirable  value  ““y be 


4.4.1. 1.5  Constraint  Representation  (C) 


Constraints  may  be  classified 
that  constraints  from  different 
uniform  temporal  constraints. 
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Temporal  Constraints 


Figure  3.  Temporal  Constraint  Categories  in  Z-Scheduler 

Z-Scheduler  diftanua.es  ■ being 

constraints  can  be  represented  in  any  of  the  categories  s T , * Task- 

below 

1. LET>EST 

2.  LET-EST  >=  Duration 

3.  ET  > ST 

4.  ET-ST  = Duration 

The  global  constraints  are  used  by  the  Truth  maintainer  knowledge  source  to  check  for 
possible  conflicts  in  the  current  schedule. 

Amnne  the  local  constraint  categories,  Z-Scheduler  is  currently  geared  tohandle  the 
local  singular  constraints  can  be  made  using  logical  relations  like  OR,  AND  . 
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Considering  the  OR  logical  connective  between  singular  constraints,  we  have  disjunctions 
ong  constraints,  within  which  we  can  specify  categories  as  shown  in  Figure  4 below. 


\ # of  tasks 

^considered 

# of  \ 
constraint  \ 
categories  \ 

One 

Many 

One 

eg.  (before  A B) 

eg.  (OR(before  A B) 
(before  A C) 
(before  A D)) 

SAME-RELATION- WITH- 
SAME-TASK 

SAME-RELATION- WITH- 
MANY-TASKS 

Many 

eg.  (OR(before  A B) 
(equals  A B) 
(overlaps  A B)) 

eg.  (OR(before  A B) 
(equals  A D)) 

MANY-RELATIONS-WITH 

-SAME-TASK 

MANY-RELATIONS-WITH- 

MANY-TASKS 

Figure  4..  Categories  of  Disjunction  Relationships 

m,aSSriZ  S i0^  constraint  representation  is  a merger  of  both  the  qualitative  and  the 
quantitative  time  constraints  and  hence  is  more  expressive  than  just  any  one  of  the 
representations  alone.  It  must  be  remembered  here  that  Z-Schec/uIer  uses  the  hybrid 
structure. °f  temporal  constraints  only  for  representation,  and  translates  these  constraints  into 

4.4.2  Performance  and  Quality  Engineering 

3™* 1 benchmarking  session  for  the  Z-Scheduler,  the  time  taken  for  scheduling 

haVin®  f°Ur  constra,nts  is  about  9 minutes.  The  70%  of  the  time  taken  for  the 
schedule  generation  was  spent  on  performing  input/output  to  the  displays  Also  for  this 

“h  SSS?" 2*  SOlu,ion.sPa«  ™ W.  or,  in  other  wortfSten  to 
was  high  and  hence  there  was  an  increase  in  the  deliberation  time  because  the  Z-SchedulVr 

are^om^offh"^  re  Promis'nS  start  (>me  and  end  times  for  each  task.  Given  below 
are  some  of  the  performance  improvements  the  author  has  made  to  fine  tune  the  Z's 
deliberation  time  to  a minimum.  e s 

Wl(h  KS;  It  is  a well  known  fact  that  greater  a databases'  size  the 
more  dme  it  takes  to  access  objects  from  it.  Similarly,  viewing  the  BlackboS  as  a colWn 
shared  database  among  the  knowledge  sources,  it  becomes  computationally  expensive  to 
access  and  add  objecls  in  ihe  Global  blackboard.  One  of  the  solutions  impleSS  .o 
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overcome  the  above  problem  was  ^fblackS required  only  by 
JXing  oto  knowledge 

initially  chosen  for  its  lntuiuve  par  P sQur®es  turned  out  to  be  functional.  This 
the  problem  decomposmon  int  ces  that  most  of  them  to  be 

functional  decomposition  of  the  knowie  g knowledge  sources  is  in  terms  of 

activated  serially.  The  mechanism  in  GEST  to ' Order  mow^  g simultaneous 

shared  and  unshared  queues.  and  KS2  belong  to  the 

execution  of  the  knowledge  sou _ ( t both  have  that  are  active  .^s^mc 

same  shared  queue,  and  at  any  instant  >n  | X 27  in  KS2.  Then  the  execution  will  allow 

H is  rule  1 1 end  rulel2  in  KS1 wd Unite .21  ” ^fand  ,hen  fire  rule22  in  KS2.  If 

rulel  1 in  KS1  to  fire,  fire  ruL  chor^H  nueue  then  order  of  rule  firings  will  be 

KSl  and  KS2  belonged  tothesameunshadq  J instance  0f  time  the  controller 

rulel  1,  rule  12,  rule  21,  rule22.  Also,  in  born  c the  rules  ready  for  firing  which 

orders  all  the  knowledge  sources  to  in^wt  wnhin  must  matych  in  the  patterns  in 

means  that  each  of  the  kno^fae'^  local  blackboard  to  identify 

its  rules  to  the  contents  in  the taSSSffioaici  can  only  be  activated  in 
the  candidate  active  rules.  Since  rn  atrhinti  conflict  resolving  in  each  KS  was  seen  to 

sequence,  .his  unifonn  ins.an.  to  of  te  KS  was  such  .ha.  any 

be  expensive  in  .emus  of  “JJSS would  be  to  increase  its  own  prionty  so  as  to 
time  one  KS  was  invoked,  the  first  act  on  wou  when  the  knowledge  source  realizes 

thwart  any  other  KS  to  get  the  process mg t -A  lso^  volumarily  decreases  its  prionty  so  as 

its  function  to  be  neanng  ““Pj®1.1®?  f rces  to  run.  The  only  exception  to  the  above 
■ "feSEJSS STSK Sler  CTrulh  Mainuiner)  KS,  since  .he  very 


to  allow  for  < 


fu  a. - the  very 

dynamic  prionty  to  the  KS  ^he  \j  this  function  is  an  alltime  one  and 

»igLr  priority  than  .he  res.  of  .he  knowledge  sources. 

3.  parthinnine  the  Blackhwri:  Ay  me  retrieval  of  objects  from  die  blackboard.  In 

SerfzShSut?w“« SS  up  the  space  using  the  frame  hierarchy  concept  provnied 
in  the  GEST  framework. 

4.  rinsing ExtendedTflpk-bfl^d^^^^til^'  time-efficiency, 

algorithms  is  the  number  of  backtracks  * S[  extended  version  of  the  task-based 

Jctd« 

SuMK 

the  conflicts,  thereby  reducing  the  number  of  backtracks. 

4.4.3  Implementation  Constraints 
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kJinwl2ST^CepS  ^3Ck  of  u 1 thc  changes  made  t0  the  working  memory  of  all  the 
knowledge  sources  during  the  execution  of  the  Z-Scheduler,  10000  blocks  of 

paging  space  is  suggested  for  an  acceptable  response  of  Scheduler SaLtitv  of 
paging  space  may  be  higher  for  an  input  containing  more  than  100  tasks  ^ ^ 

Iso  it  is  suggested  that  between  few  runs  of  the  Scheduler  an  immediate  GC  be  performed. 

5.0  DESIGN 


5.1  ARCHITECTURAL  DESIGN 

The  architecture  for  the  Z-Scheduler  is  shown  in  Figure  5. 
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^nPTWARE  ARCHITF™1RE  of  7-schepuleb 


INPUT  TASKS  AND  CONSTRAINTS 
FROM  TASK  GENERATOR  / PLANNER 


OUTPUT  CONSTRAINTS 
FROM  Z-SCHEDULERTO 
TG/PIANNER 


Figure  5.  Software  Architecture  of  Z-Scheduler 


be  modular  and  easily  extendible.The  components  of  ZSdSufe^  for  lts  components  to 
sources  and  each  component  represents a SL t 81X5  caIled  Pledge 

is  modular  and  is  called  on  an  "as-needed"  basis  hwhf^U  ing  pr?9css-  Z-Scheduler  itself 

theindivid“al  tol 

been  extended  to  apply  to  mulSle  shar?abi  rf d scheduIl?g  decomposition  has 

faster  performanceTncorporating  Z-SchedLer  intoSe  rcschcduling  “d  tuned  for 

model  to  reason  about  model  ^ows  the 

thereby  enhancing  the  predictive  capability  of  the  MIDAS  tS  Sysk®  h°rizon)’ 

probi™^ 

we  can  fit  our  scheduling  problem  into  the  cateeo™  of  BaSCd  UpP" the  nature  of  *e  inP“t 
(Ben-Arieh,  1985).  It  is  dynamic  tecause  the  S nf  n}ulu-staSe  ro^g  problem 

events)  over  time  It  is  muhSL  tecalse  all  fhl f T?  fks  £hange  (due  ,0  ^expected 
homogeneous  (in  the  sa™S  Muled  may  not  ** 

abstract  tasks  may  have  to  be  expanded  further  bv  thTsrh^H°?  strruc.ture);  bence,  the  more 
beca.se  .here  may  * situations  1^2^^ 

problems  (S^O^s  a?^87^i,p88*lCH^1^>vlde  eftcctJ  ve  solutions  for  such  complex 

sSa^ 

frome-buseti  schedule  ^presentation,  and  problem  spacl'S^XtfSf  ^ 

5.1.1  Design  Approach  and  Tradeoffs 

Scheduling  problems  are  classified  based  on  the  following  categories. 

u^is^SS5SS?ft- 

3 Assessed* by  the  talks.5  If^  S baSed  °2 the  execution  resources 

it  is  a queuing  problem  (queue  for'ihat pmS^iSoJ^)*  ffa^SSib? the" 

than  ” " a routingP-blem  (routetaTalremate 
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The  fira  categorization  ^ also  J* 

£8E3£H^^ 


eauie.  n nas  dccu  vi^<u  uw»*  ^ 

e (dynamic)  scheduling  problem 

as$umpt£gsi; 

*SS£!oI^^ 

future  when  this  assumption  is  lifted  then  problem  becomes  a routing  one. 

eiSSSSSSr 

sss^MsS**. 

problem.  Knowledge  ^^SSS^iSSi^  of  possible*  schedules 

problem  as  a constraint  satisfaction  problem  (CSF).  L*r  nas  a smicuuc 
Figure  6 below. 
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CONSTRAINT  SATISFACTION  PROBLEM 


Figure  6.  Constraint  Satisfaction  Problem 


CSP  contains  a pool  of  variables  that  have  a corresponding  domain  of  values  and  the  aim  of 

^cg°n?m  *5“  SOlveS  t*  ?SP  is  t0  ass'g"  for  all  'he  viiables  a value  fromus 

dr  SUifh  11131  all  the  assiSnrnents  do  not  violate  any  constraints  between 
th^  IT  tb  CS  defined  !n  |he  constraint  pool.  Whenever  a problem  is  modeled  in  this  manner 
InVthLlT  SPiaCCi  ls.dlvlded  'nt0  subspaces  and  the  local  solutions  are  found  for  each  space 

Si^J^^i“tt^S-!l^^,read^I.aIong  IO  get  lhe  global  soIudon-  The  method  in  which 
me  problem  space  is  divided  is  crucial  to  the  efficiency  of  the  algorithm  that  solves  the  r<5P 

One  of  the  metrics  for  the  CSP  algorithms  efficiency  is  the  numtorfbSSS^SSS: 
lower  the  number  of  backtracks  better  the  algorithm*  In  the  Z Scheduler  weTate 
more  recent  task-based  scheduling  to  partition  the  scheduling  space  since  the  aleorithm  that 
can  solve  the  subproblems  takes  cant  of  the  interaction  betwlTSabK£  K 

“P  front  and  hence  minimizes  on  the  backtracks  to  handle  the  conflicts  The 
SSSh’ISSSlTiSSSll?  SelCC,W  md  Allocator  knowledi^Mjijice 
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5.2  DETAILED  DESIGN 

5.2.1  Detailed  Design  Description 

these  knowledge  sources. 

5 2 11  Constraint  Solver  Knowledge  Source 

•n*  Constrain,  Solver  knowledge  source  over, 

uansforms  the  given  constr *»» the  actual  schoduhng 
Preprocessing  here  means  that  the  Pr°c,^  p h run  0f  the  Z-Scheduler.  The  constraint 

sfes; phas'  ^ pareal 

dependency  graph  (PDG)  generation  phase. 

During  the  constratn,  onlering  phase,  the  input  constrain,  list  is  ordered  accordrng  to , e 
following  gnjB*  to  their  “^.S^KSspecifted  in  earlier  step. 

."thecSrMSS 

6 

The  creation  of  PDGs  is  one  of  thef™"^  order-  This 

created  by  inferencing  on  the  set  of  tempo  , . . constraint  frame  one  at  a time 

process  involves  accessing  the  slot  values  c°nta  EST-value,  LET-value, 

Shading  the  PDGs.  Concurrently  it  «£««***  [Krnporal  transform  tables.  The 
ST-values  and  ET-values  by  perfommg  P ’ „ attributed  to  the  work  by  Mark 

notion  of  applying  constraints  one  at  a tune ^ g J*  Additionally,  during  the  PDG 
Stefik  in  MOLGEN  and  is  temied  constrai  p g translated  into  point-based  filters 

Iabl“  as  shown  below' 
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\ Constraints 
\ 

\ 

Task-A’s  \ 
Slot-values  \ 


Before 


Example 

EST-value 

EST-con  strain  ts- 
unscheduled 

EST-constraints- 

scheduled 

LET- value 


(Before  A B) 


I 


During 


Overlaps 


(Meets 


(Starts 

I 


(During  A B) 


I (Stans  A B) 


1 I 

I (Overlaps  A B)  l(MeclsAB) 

I I ; 

bT(A, HSTtA).  EST^DCA,  IeSTW  . 

I I ^ } | I EST(B) 

|EST(A)  = ST(B)CST(A)=STW  I EST(A)  = ST(B)-D(A)  'lEST(A)  = 

I I I 1 ST(B) 

' 

' 


I j , ( LET(B> 

II  I l r\/m  — 


LET-constraints 
scheduled 


Stan-time-value 


|LET(A)  = ST(B)  ILET(A>=  ET(B) 
-1  I -j 


Start- time-con  s- 
-traints 


End-time- value 

End- time-cons- 
traints 

Duration 

constraints 

PDG -constraints 


1 LET(a>-st<b) 


IST(A)>ST(B) 


j D(B)  +D(A) 

ILET(A)  = 

I ST(B) 
l+D(A) 


IST(A)  < ST(B)  IST(A)  = ST(B)-D(A) 

I 
I 

ET(A)  < ST(B)  IET(A)  < ET(B)  ET(A)>ST(B)  ET(A)  = ST(B) 

ET(A)<ET(B)  ■ ’ 

1 I 

10(A)  >=  D{B)  t-2 1 D(A)  >=  2 
1 I 

' I 

JP(A)  - P(B)  IP(A)  = P(B)  IP(A)  = P(B)  - 1 
where  P( A)  = position  of  task  A in  its  PDG.  I J 


I ST(A)  = 
I ST(B) 


'P(A)  < P(B) 


ET(A)  = 
IST(B)+D(A) 


P(A)  = P(B) 


C8  Tab'£  '•  TemP°ra'  Tr*”^7pa7a^”di‘^“ 
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\ Constraints 
\ 

\ 

Task-A's  \ 
Slot-values  \ 

1 

1 

Equals 

1 

1 

Example 

1 (Equals  A B) 
l 

EST-value 

1 

I 

EST-constraints 

-unscheduled 

1 EST(A)=EST(B) 

1 

1 

EST-constraints 

-scheduled 

1 EST(A)=ST(B) 

l 

I 

LET-value 

1 

I 

LET-constraints  ILET(A)  - LET(B) 

-unscheduled  I 

I 

LET-constraints  ILET(A)  = ET(B) 
-scheduled  i 


Start-time- value 


Start-time-cons- 
- train  ts 


ST(A)  = ST(B) 


End-time-value  I 
I 

End-time-cons-  IET(A)  = ET(B) 

-traints  * 

I 

Duration-const-  ID(A)=D(B) 

-raints  1 

I 

PDG -constraints  1P(A)  = P(B) 

I 


IStarted-by 

I 

I 

l (started-by  A B) 

1 EST(A)=EST(B) 

\ EST(A)=ST(B) 

lLET(A)=LEr(B)-D(BHD(A) 

CLET(A)=ST(B>+D(A) 

I 

I 

I 

I 

IST(A)=ST(B) 

I 

I 

I 

I 

ET(A)=ST(B)+D(A) 


I 

Finished-by 


(finished-by  A B) 


EST(A)=EST(B)+D(B)-D(A) 

EST(A)=ET(B)-D(A) 


1LET(A>=LET(B) 

I 

I 

ILET(A>=ET(B) 

I 

I 

I 

I 

1ST(A>=ET(B)-D(A) 

I 

I 

I 

I 

ET(A)=ET(B) 


Table  l(cont). 


Temporal  Transform  _ llncondi.ional-Re.ativ,  Constrain, 
Propagation 


\ Constraints 
\ 

\ 

Task-A’s  \ 
Slot-values  \ 


lAfter 


I 

^Contains 

I 


JOveriapped-by 


Example 

EST-value 

EST-constraints 

unscheduled 


KAfter  A B)  ((Contains  A B)  KOverlapped-by 


AB) 


EST-constraints 

scheduled 


EST(A)=EST(B)  EST(A)  = EST(B)I  EST(A)^EST(B) 
I +D(B)+ 1 1 + D(B)+ 1 -D(A)  I +D(B)+1-D(A) 

■ J I otherwise 

1 j EST(A)=ST(B)+1 

EST(A)=ET(B)I  if  D(A)<D(B) 
+1-D(A)I  EST(A)=ET(B)+ 1 
i -d(a) 


EST(A). 


= ET(B) 
+1 


IMet-by 


Finishes 

I 

I 


KMet-by  A B) 


EST(A)  = EST(B) 
+D(B) 


EST(A)  = ET(B) 


((finishes 
I A B) 

I 

I 

EST(A)=  - 
I EST(B) 

I +D(B) 
l-D(A) 

I 

EST(A)=  - 
I ET(B) 
l-D(A) 


LET-value 

i 

1 

J j EST(A)=ST(B>f  1 

1 

i 

1 

1 

LET-constraints 

LET-constraints 

scheduled 

Start- lime- value 

1 

1 

1 

1 

1 

1 

1 

{ 

1 1 

ILET(A)=LET(B)I  LET(A)  = LET(B) 
l-D(B)-l+D(A)  1 -1  +D(A) 

1 1 

EET(A)=  ST(B)  1 LET(A)=ET(B)-1 
j -1+D(A)I  +D(A) 

1 1 

i 

i 

ILET(A)=  LET(B) 
(+D(A) 

1 

(LET(B)  = ET(B) 
j +D(A) 

| 

i 

i 

EET(A)= 

1 LET(B) 

1 

EET(A)=  - 
1 ET(B) 

1 

Start- time-cons- 
traints 

End-time-value 

l 

IST(A)  > ET  (B) 

1 1 

IST(A)  < ST(B)  1 ST(A)  < ET(B) 

J J ST(A)>ST(B) 

1 1 

1 

IST(A)  = ET(B) 

1 

1 

1 

1 

IST(A)=  - 
ET(B) 

l-D(A) 

1 

End-time-cons- 

Duration 

constraints 

1 1 
ET(A)>ET(B)  1 ET(A)  > ET(B) 

1 i 

ID(A)  <=  D(B)-2  1 D(A)  >=  2 
1 1 

1 

*ET(A)^ET(B) 
1 +D(A) 

1 

i 

1 

1 

IET(A)=  traints 
I ET(B) 

1 

1 

PDG-constraints  I 

i 

1 

P(A)  > P(B) 

1 1 

(P(A)  = P(B)  1 P(A)  = P(B) 

i ! 

i 

1 

IP(A)  = P(B)  + 1 
1 
1 

1 

1 

*P(A)« 
1 P(B) 

1 

Table  l(cont). 


Temporal  Transform  - Unconditional-Relative 
Propagation 


Constraint 
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\ Constraints 
\ 

\ 

\ 

Slot- values  \ 

I 

1 

1 Start-at 
1 
1 

i • 

i i 

1 Start-by  1 End-at 

1 1 

1 1 

1 

1 

1 End-by 
1 
1 

Example 

l(Start-at  A 5) 
1 

KStart-by  A 6)  l(End-at  A 7) 
1 > 

l(End-by  A 8) 
1 

EST-value 

1 

i i 

i i 

1 

1 

LET-value 

1 

1 

1 

1 LET  = 6 + D(A)I 
1 1 

1 LET  * 8 
1 

Start-time-value 

1 5 
1 

1 1 
1 1 

1 

1 

End-time- value 

1 

1 

1 1 1 
1 1 

1 

1 

.c8.Table  2 


Temporal  Transform  - Unconditional  Absolute  Constraint 
Propagation 


Page  B-27 


The  PDG  generation  stage  includes  taking  one  constraint  at  a time  and  inserting  it  into  each  of  the 
tasks  and  at  the  same  time  updating  the  partial  dependency  graph.  The  process  will  result  in  strinc 
of  partial  dependency  graphs  being  formed.  The  function  that  forms  the  PDG  takes  in  two  tasks 
say,  task- A and  task-B,  and  PDG  that  the  tasks  belong  to,  say  PDG-1  and  PDG-2  and  works  in 
the  manner  as  shown  below.  ana  WOTKS  in 


CASE-1  : 

CASE-2: 

CASE-3: 

CASE-4: 

CASE-5: 


When  PDG-1  = PDG-2  = nil 

form  a new-PDG  frame  named  PDG-AB  with  the  task-order-list  as  (A  B). 
insert  into  member-PDG  slot  of  each  of  the  tasks  the  new-PDG  frame 
l.C.,  i Dvj‘AB. 

When  PDG-1  = nil  and  PDG-2  = PDG-B 

pokuto^DG  B*0  PDG'B  ^ up<kte  Task-A’s  member-PDG  slot  to 

Insert  Task-A  into  PDG-B 

When  PDG-1  = PDG- A and  PDG-2  = nil 

Insert  Task-B  into  PDG-A  and  update  Task-B's  member-PDG  slot  to 
point  to  PDG-B. 

Insert  Task-B  into  PDG-A 
When  PDG-1  = PDG-2  = PDG-AB 

notriuP^ate  t*1c  member-PDG  slots  in  the  respective  task  frames 
When  PDG-1  = PDG-A  and  PDG-2  = PDG-B  and  PDG-A  '=TdG-B 
Merge  PDG-A  and  PDG-B  into  a new  PDG  = PDG-AB 
update  the  member-PDG  slots  of  task- A and  Task-B 


5.2.1. 2 Task  Selector  Knowledge  Source 

task'based  scheduling  for  partitioning  the  scheduling  problem  space 
Task-based  scheduling  is  a two  step  process.  The  first  step  involves  the  selection  of  a single 

Vk3Ck  716  ?ecorJ?  *teP  involves  the  commitment  of  a resource’s  time  slice 
all  alternative  resource  allocations  for  that  task  and 
selecting  the  best  allocation  based  on  certain  decision  metrics.  Each  of  these  two  steos  is 
carried  out  in  separate  knowledge  sources,  the  first  being  implemented  by  the  task  selector 

source^6  S°UrCe  **  SCCOnd  1)61116  Carried  out  by  the  resource  allocator  knowledge*^ 

VSCCtinf- Visks{oT  resource  allocation,  is  called  the  Task  Criticality  (TC) 
measure.  Task  Cnticality  in  its  most  simple  form  is  the  ratio  of  the  execution  time  to  die 
window  duration  - that  is,  it  is  a measure  of  the  slack  of  a task.  The  more  Sack  on  the 
h^if  task  cnticality.  If  TC  equals  1 , then  it  means  that  the  task  is  very  critical 
because  it  has  only  a single  solution  for  the  task  and  does  not  have  alternate  solutions 

thr  tauSk]b?Sed  S?bCuUling  (Rossi*  Keng,1989)  can  only  handle  single  resource 
rrhSUeS  t3SkS  ?a*  °-e  asijlgle  machine),  we  had  to  extend  thil  approach  to 
“1«*hc  multi-resource  domain  (dictated  by  the  nature  of  HQ  tasks).  Each  of  these 
resources  were  shareable  (at  any  moment  in  time,  a resource  can  handle  more  than  1 task  bv 
giving  x-umts  of  resource  to  task-1  and  y units  of  resource  to  task-2).  Hence  the  TC  ^ 
calculations  had  to  be  extended  to  handle  the  multi-shareable  resources. 

The  task  criticality  extended  in  the  Z-Scheduler  is  a measure  having  three  factors  slack  an 
the  task,  the  user  defined  priority  of  the  task  and  the  resource  requirement  for  the*  task  The 

b^ns^  a£f  ied  m tbe  task  selector  knowledge  source  is  that  the  task  with  highest  task 
cnticality  becomes  the  candidate  for  resource  allocation.  6 
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Along  with  the  task  to  be  scheduled,  the  heuristic  had  to  select  a resource  that  needed  to  be 
focussed  upon  and  the  logic  is  oudined  below. 

Task -criticality  :»  TC  «.  ^^^TcLET.' Ism 

Task-selected  ::=  t ::=  Task  with  Max  TC 
Resource-selected  ::=  R ::=  Max  [V  A C P]  of  t 
TC-for-tasks-that-use-resource-R- 

in-the-window-duraUon^of-task4pr_or^  # R „ (duration/LET.EST)) 

5 2.1.3  Resource  Allocator  Knowledge  Source 

Similar  to  calculating  the  ideality  for  the 

scheduled.  This  metric  is  called  the  resource  compeution  measure. 

Resource-Competition-ima.Iime-pmod  ^ 

Resource-Competitiomfor-solution^^e  (RQs^ 

Rp«iurce-allocated-to-task-t  ::=  Min  (RC)s 

::=  The  solution  selected  is  the  one  having  the 
minimum  resource  competition 

Hence  if  there  are  alternate  solutions  to  allocating  the  resource  to  a task  then,  Z-Sdieduler 
2d5*e  solutior^thaThas  the  least  impact  on  the  rest  of  the  tasks,  i.e,  it  selects  the  tune 
period  of  the  resource  that  is  least  competitive. 

5. 2. 1.4  Constraint  Propagator  Knowledge  Source 

£3a SU td^onunity  by  propagadng  the  temporal  constramts  m the  paroal 
dependency  graphs. 

Thr  pffert  of  allocating  time  units  of  a resource  to  a task  is  propagated  in  two  stages.The 

between  all  the  PDGs  that  have  tasks  that  overlap  the  time  units  of  the  resource  assigned 
and  is  defined  as  the  Inter-PDG-propagation  stage. 

5 .2.1.4. 1 Intra-PDG-propagation 

3SS£s3Si3sffi3E=s 
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S^K2,.alf>  u.se*  ^grouping  below.  The  constraints  belonging  to  the  preceding-tasks 
the  task  just  scheduled  are  first  propagated,  followed  by  the  constraints  belonging  to  the 

™kS  “d  h?1*  *'  C0"S,rato,S  b"°n«i"6  “ *0  concJSTisS  ST® 

propagated.  The  reason  for  doing  so  is  that  the  preceding-  and  the  succeeding-tasks  reduce 
the  window  of  opportunity  much  more  than  the  constraints  belonging  toTSS 


The  following  task  categories  are  formed  by  the  given  constraints  and  are  as  shown  below. 

Preceding-tasks  = tasks  that  are  constrained  by  the  ’Before  or  Meets  constraint  types. 

Succeeding-tasks  = tasks  that  are  constrained  by  the  ’After,  Met-by  constraint  types. 

Concurrent-tasks  = tasks  that  are  constrained  by  the  Overlaps,  Overlapped- by,  During 

Contains,  Starts,  Started-by,  Finishes,  Finished-by,  and 
Equal  constraint  types 


The  windows  are  initially  calculated  for  the  first  time  immediately  after  the  constraint 
solving  knowledge  source  has  finished  execution.  Incremental  OR  algorithms  calculate  the 
initial  window  of  opportunity  for  all  the  tasks.  The  window  of  opportunity  is  represented 
as  a time  slice  that  is  bound  on  the  left  by  Earliest  Start  Time  (EST)  value  and  is  bound  on 

finrW 1 Kt1 LT^El)d  Time  ^ET)  value-  ^ EST  fOT  <*ch  task  is  calSlaSd by 
ctart  rtf  t^e  •tasks  that  do  have  ^ Preceding  tasks  and  assigning  to  the  EST  value  the 

the  tasks Ch3ining  t0  thC  EST  °f  ^ rest  of 


Initial  window  calculations: 


Forward  chaining  to  calculate  the  EST 
For  all  tasks  with 


Preceding-tasks  = nil 

If  ST  or  ET  = nil  i.e  they  are  not  scheduled  yet 
Then  EST  = start  of  the  given  temporal  horizon 
Loop  for  all  succeeding-tasks  of  task 

apply  constraint-propagate-unscheduled  (test  only  EST-constraints) 
else  Loop  for  all  succeeding-tasks  of  task 

apply  constraint-propagate-scheduled  (test  ST  constraints  and  if  fails 

then  EST-constraints) 


Backward  chaining  to  calculate  the  LET 

For  all  tasks  with 

Succeeding-tasks  = nil 

If  ST  or  ET  = nil  i.e  they  are  not  scheduled  yet 
Then  LET  = end  of  the  given  temporal  horizon 
Loop  for  all  preceding- tasks  of  task 
apply  constraint-propagate-unscheduled  (test  only  LET-constraints) 

Else  Loop  for  all  succeeding-tasks  of  task 

apply  constraint-propagate-scheduled  (test  ET  constraints  and  if  fails 

then  LET-constraints) 
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The  windows  am  updamd  after  the  execution  of  <he  msoume  allocation  ftage  using  a 
recursive  algorithm  as  shown  below. 


Loop  for  each  cOTsuainwi  hyJhe  just  scheduled^  formulas 


LCT  vXet  g^:  ^eWCre 

ET  or  ST  then  the  ynpstraint-propaganpn-sgncqmfifl 
functions  is  explained  below. 


f|kn<ifraint-propaga''nn~!>g^,J^ 

For  each  Predecessor  in  the  preceding-tasks 
If  predecessor  is  scheduled 

5“  identify  ST-filrer  tira.  mla.es  dm  scheduled-u.sk  and  dm  Predecessor 
ft  STo/alue  changes  for  tile  Predecessor  recursion!! 

^JZXSSBEZSSS^  ami  dm  *— « 

tf  ET-value  changes  for  the  Predecessor  , -recursion' ' 

ApplIfEST^^^chmgesfOT^eI^w^^^  ^ 

Identify  LET-filter  that  relates  the  scheduled-task  and  the  Predecessor 
Apply ' ^jiT-value  Ganges  for  the  Predecessor 

Then  call  constraint-propagation-unscheduled 

For  each  Successor  in  the  succeeding-tasks 
If  successor  is  scheduled 

Identify  ST-fdter  ,ha,  mla.es  the  scheduled-task  and  dm  successor 
AddIv  that  filter 

If  ST-value Ranges ***** 
Identify  ET-filtl *at  relates  the  scheduled-task  and  the  successor 

Applet value  changes  for  the  successor  , -recursion' 

Then  call  constraint-propagation-sch^ul^  r, 

Identify  EST-filter  that  relates  the  scheduled-task  and  the  successor 
Apply  that  filter 
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EST-Value  changes  for  the  successor 

Identify  LET  constraint-propagation-unscheduled 

Apply 1 rc,atCS  1,16  •**»**»*  the  successor 
If  LET- value  changes  for  the  successor 

Then  call  constraint-propagation-unschcduled 

For  conc-task  in  the  concurrent-tasks 
If  conc-task  is  scheduled 
then  do  nothing. 

rclaKS SChedul nd  conc-task 
If  ST-value  changes  for  the  conc-task 

If  ET-value  changes  for  the  conc-task 

If  EST-value  changes  for  the  conc-task 
Identify  LET  fihJf  Ao?”  “"straint-propagation-unscheduled 
Apply  thSS  re  68  thC  "*******  and  the  conc-task 
If  LET-value  changes  for  the  conc-task 

cn  call  constraint-propagation-unscheduled 

£QnStraint-ProDaeaHnn-,mr^r||1|.fj 

FTfn^H?CdeCC?SOr.in  the  P^eding-tasks 
lr  predecessor  is  scheduled 

then  do  nothing. 

^ th'  “"^hedulcd-Mt*  and  the  Predecessor 

If  EST-value  changes  for  the  Predecessor 

Identify  LET  filfpr?h^^°,nSdTnt'Ptropagadon'unscheduled 
Apply  tha^fil tar tCr  **  ^ schedu^-task  and  the  Predecessor 

If  LET-value  changes  for  the  Predecessor 

Then  call  constraint-propagation-unscheduled 

For  each  Successor  in  the  succeeding-tasks 
If  successor  is  scheduled 
then  do  nothing. 

Appi%mSfiltSter  th3t  re,atCS  **  unscheduled-task  and  the  successor 
tf  EST-value  changes  for  the  successor 

K-  LET-value  changes  for  the  successor 

Then  call  constraint-propagation-unscheduled 


Page  B-32 


For  conc-task  in  the  concurrent-tasks 
If  conc-task  is  scheduled 

Ely  EST-fiUer  iha.  relates  the  unscheduled-task  and  the  conc-task 
Apply  that  filter 

If  EST- value  changes  for  the  conc-task 

Then  call  constraint-propagation-unschedulea 

Identify  LET-filter  that  relates  the  unscheduled-task  and  the  conc-task 
Apply  that  filter 

If  LET-value  changes  for  the  conc-task 

Then  call  constraint-propagation-unscneaulea 

5.2. 1.4.2  Inter-PDG-propagation: 

let  EST-temp(task)  = EST  (task) 

^’p' andStt-peak  in  peaks  for  all  odd  numbered  peaks 
If  EST-temp  (task)  < = ST(peak) 

Then  If  LET-temp  > ST(next-peak) 

Then  MAY-BE-EST-n  = EST-temp 
MAY-BE-LET-n  = ST(peak) 

MAY-BE-EST-n+1  =ET(peak) 

£ MAY-BE-LET-n+l  = UtT-temp  (^LET-tentp  >ST  (next-peak, 

MAY-BE-EST-n  = EST-temp 
MAY-BE-LET-n  = ST(peak) 

MAY-BE-EST-n+1  = ET(peak) 

MAY-BE-LET-n+l  = ST(next-peak) 

EST-temp  = ET(next-peak)  ~T  ineakl 

Else  If  LET-temp  > ET  (peak)  ;;;  »■«■.  EST-temp  > ST  (peak) 

Then  If  LET-temp  < ST(next-peak) 

Then 

MAY-BE-EST-n  = ET(peak) 

MAY-BE-LET-n  = LET-temp 
EST-temp  = ET(next-peak) 

£|se 

No  Solution  - hence  backtracking  required. 

Find  the  validity  of  each  of  the  windows  by  checking  if 
LET-x(task)  - EST-x  (task)  >=  Duration  (task) 

If  false,  then  drop  that  window  duration  from  the  list 

possible-windows.  . - 

Among  the  valid  windows  select  the  largest  one  as  the  prime 

CalUnOra-DG  propagation  using  the  largest  window  duration  as 
the  changed  reference. 

5.2.1.5  Truth  Maintainer  Knowledge  Source 
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™l?Z*$?nmner  kno^ledgc  is  a hiSh  level  monitor  of  the  constraint  propagating 
Kf;  Whenever  a conflict  occurs  during  the  process  of  constraint  propagation! 

After  halrin^th  " 0f  ^ knowlcdge  sources  by  sending  a message  via  the  blackboard 
£ t£,  ? ^ PT?eSS  that,w?s  "sponsible  for  the  conflict—  an  exLple  of  the 

msk  * thTTmrh  mS°"  °f  *5? taSk  becomes  smaller  than  the  estimatedduration  of  the 
tosk  — the  Truth  Mainainer  attempts  to  find  the  reason  for  the  conflict’s  occurrence  and  on 
findrng  it  out  it  resets  all  tire  knowledge  sources  to  that  point  in  time  dm 
conflict.  In  other  words,  the  Truth  Marntaxner  is  built  on  the  principle  of  chronological 
backtracking,  where  the  amount  backtracking  is  to  the  previous  decision  point.  ** 

aredescribed  as  any  violations  of  the  global  constraints  as  described  in  the 
ection  on  constraint  representation.  For  example,  the  global  constraint  1ET-EST  >= 
Duration  when  violated  means  that  the  window  of  opportunity  for  a task  is  less  than  the 
al^rifh  of  the  tosk  henCe  making  the  task  impossible  to  schedule.  The  truth  maintenance 
algorithm,  applies  all  the  global  constraints  whenever  any  task's  EST-value,  LET-value 
f-value  or  ET-value  changes  during  the  constraint  propagation  stage.  Currently  Z-  ’ 
anH  hlner  pHerforms  the  chronological  backtracking  mechanism  to  handle  the’conflicts 
w d°^  n0t  b^/Hy  domain  knowledge  to  relax  the  constraint  without 
backtracking.  One  of  the  future  extensions  of  the  Z-Scheduler  is  to  include  domain 
dependent  strategies  that  handle  the  conflicts  without  backtracking,  which  happens  to  be  a 
more  psychologically  valid  method  for  handling  the  conflicts. 

5.2.1. 6 Z-Scheduler  Display  Design 

DrtxiSs  °S  nfCth^dH  SCIiibeS  SC^er^  typesJof  dspfcys  used  in  visualizing  the  scheduling 
JV  f KaCu  °U  these  d}sPl*ys  m distinct  and  can  be  implemented  separately  The  onlv  g 

teSSaES?* ,S  *“  CliCtas  - a P™"  " 4lay  A 


Please  note  that  : 

Flavor  classes  appear  in  italic  type, 
presentation  types  appear  in  bold-italic  type. 
Function  and  variable  names  appear  in  bold  face. 

5.2.1.6.1  Task  Stack  Display 


pus  display  shows  the  order  in  which  the  tasks  will  be  performed.  This  display  will  be 
rnntain^  ^ W update  itself  any  time  the  scheduler  reorders  the  tasks.  The  window  will 
En  ,r,  ^°n  object  for  each  task.  A task  object  wUl  look  like  a playing  card  with  a 

the  top.  The  task  objects  will  stack  on  on  top  of  another  to  show  the  ttfsk  ordering. 


I Task  C I 

I 

I Task  B I— 

I 

I Task  A I- 
I I 

I I 

task-stack-window:  window  flavor  which  will  be  used  for  task  stack  display, 
task -card:  presentation  type  for  card  like  display  of  task  object 
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♦default-task-stack -character-style*:  Default  £»- » V*  - — “ “* 
(update-task-stack 

EISS"  ^t  SF«hCtioS  wilt  remain  tughlighted  fur 
highlight-time  seconds. 

5.2. 1.6.2  Task  Frame  Display 

This  is  a simple  display  which  shows  the  slot  values  of  a task.  It  will  look 
something  like  this: 


Name:  Task  A 
Agent:  ... 

Resource: ... 

object-frame-window:  window  flavor  for  pop  up  windows  showing  slots  of  frame. 

-sssssssasss ir:i 

•default-frame-character-style*:  default  font  used  to  print  slot  values. 

(show-task-fr^te.task^uperior^  st^ie.dejauj(_frame^|iaractcr.S{yle*)) 

pops  up  a temporary  *S*3£E5*^  Wh'n 

““  window  (can  be  either  color  screen  or  mam  screen), 
(get-task-slots-for-display)  returns  a list  of  slot-printer-pairs  (see  description  below). 
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the  pop-up  window  (can  be  either  color  screen  or  main  screen). 

(get-constraint-slots-for-display  reference-task  related-task) 
returns  a list  of  slot-printer-pairs  (see  description  below). 

The  generic  function  for  popping  up  a window  showing  the  slots  of  a frame  is 

C“°"  CaIW  inKn,a"y  show-task-frame  aad 

(print-frame  (frame  slot-printer-pairs  superior 

&optional  (character-style  *default-frame-character-style*)) 

slot-printer-pairs  is  a list  of  pairs 
(slot-name  printer) 

printer  specifies  how  to  print  the  slot  value 
it  must  be  one  of 

(non-mouseable-print  default-print  constraint-print  pdg-print) 
non-mouseaWe-print  slot  value  will  not  be  mouseable. 

5.2. 1.6. 3 Dependency  Graphs 

«ec„°Zin  pjjT  ^ Tf  “T?*  d'P'"d'“y  relationships 

ocatr  before  Task^and  Kk  C P ’ ^ bd°W  Shows ,ha' Task  A 

Task  B 


Task  A 


Task  C 

nodes  in  partial 

♦default-color-cycle*.  list  of  colors  we  cycle  through  when  drawing  links. 

pdg-task-node:  presentation  type  for  nodes  in  the  graph 

they  will  be  shown  as  an  oval  with  a label  in  the  middle. 

V the  mouse  on  this  presentation  type  will  pop  up  a window 
describing  the  task  object  (see  "Task  Frame  Display"  above™ 

(sho».d^denep.graph  pdg  &key  streare  (characrer-ayle  -default-task-node- 

returns  a window  containing  the  partial  dependency  graph.  If  you  do  not  soecifv  * 
steam  you  11  be  prompted  with  the  mouse  for  the  window  siSd  * 

pdg  is  an  object  which  contains  a list  of  the  tasks.  Task  objects  contain  a 
succeeding-tasks  slot  which  contains  a list.  Each  element  in  the  list  is  either 

1.  (task)  we  draw  a single  thickness  line  to  the  task  node 
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2'  ££  each  task  node 

3\Sta;atoS!in;“«cUsknode 

all  succeeding  tasks. 

Task  objects  also  contain  a concurrent-tasks  slot  whose  value  is  either  nil 
or  a list  of  tasks. 

To  construct  the  panial  dependency  graph  »elook«|he  -""ag*  ** 
SKSSSSSS  sbMandconsmict  tftose  links  (drawing  plain  lines). 


task  A 

succeeding-tasks:  ((B)  (C)) 

task  B 

succeeding-tasks:  ((D)) 


Task  A 


TaskB 


Task  C 


TaskD 


task  C 

succeeding-tasks:  ((D)) 

task  D 

succeeding-tasks:  nil 

5.2. 1.6.4  Scheduling  Chart  (Gantt  Chart) 

This  display  is  a dynamic  display  showing  the  scheduling 

like  a line  with  arrows  on  the  enas  V 6 

s.t*  ■ sr 

rectangle  (===  in  diagram). 

Stliemtog^ 
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The  display  will  look  something  like  this: 


DG1I 


< > 


< > 


DG2I 


> 


■ ■ — - • i 

mouse^Kide^^j^o^wiH  mS^iT^app^Cntire  Gantt  Chart  Cliddngthe 

unsched-task-box:  presentation  for  an  unscheduled  task  wirhin 
a partial  dependency  graph  It  will  look  lik/»  an  r ”in-  «. 
the  ends.  **e  an  ^uc^c  ^me  with  arrowheads  on 

Clicking  left  on  this  presentation  type  will  non  Ud  a 

window  describing  U*  usk  objec,?see  Disptay"  above) 

seW.to^a^bn.  for  schemed  .ask,  WiU  ,ook 

presentations  •character-style*:  character  style  of  dependency. graph 
•default-task-load-character-^tvfe^rh^1111  ^ , 

when  you  are  overlaying  a load^m ^on^Gaitt^hm  ^ ^ ***  ^ 

&key  -are  load-type 

■SK 

tekSIi«tsSt If  iS  etCd  tim?  for  each  task  in  each  P dg  from  the 
we  overlay  a load  plou£  Ms  ^JJ]atlve-load-points  are  supplied 

(modif^schedule-window  task  schedule-window) 

times  for  a tasl^  Updates  pr^  toe  changes?*"  ^ 

(schedule-task  task  schedule-window) 

The  scheduler  must  call  this  function  when  it  schedules  a task 
Changes  presen.anon  for  .ask  ,o  be  a r dwS35  pSnation. 
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(highUght-current-tasktask  it  begins  io  work 

m scheduling  ^und  the  presentation  for  task. 


the  same  time  frame  as  in  the  Gantt 


5.2.1.6.5  Visual  Loading  Plot 

This  display  shows  a plot  where  the  x-axis  is 
scheduling  chart  and  the  y-axis  is  the  load. 

Each  task  object  has  slots 
st- value:  start  time 

AC  P)  list  containing  visual,  auditory,  cognitive  and  psychontow  loads. 

For  each  task  we  put  a circle  on  the  plot  showing  (time,  load).  The  circles  will  be  die  color 
specified  in  the  color  slot  of  the  task  object. 

Also  we  construct  a line  which  shows  the  cumulative  load  for  all  tasks  a,  each  htne  „ck. 
number* of  tick  marks  etc.  and  replot  the  points. 

window  describing  the  task  object  (see  lasKrramc  y 

■■-ssss  — to 

be  defined). 

•SKS^  - « ■>'»•  '“indow 

(show-load-plot  task-list  cumulative-load-points  load-type 

^xisScter-style  *default-x-axis-character-style*) 
(y-axis-character-style  *default-y-axis-character-sty  e ) 

(axis-color  * default- axis-color*) 

(axis-label-color  *default-axis-label-color  ) ~r*W 

(load-line-color  *defauh-cumulative-load-line-color  )) 

If  stream  is  provided,  displays  load  pbnn  * sach  task  are 

SSSS  Potions.  Points  in  cumulative-, oad-po.n.s 

areSpresented  as  cumulative-point  preseutattons. 
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Color: 


variable  to  b^Ta^e^M^o^  y°H  dl?,gc  ^ defauIt  charactcr  style 

the  color  variables.  y d USe  0n  black  80(1  wh,tc  screen.  You  must  also  set 

^ See  the  function 

***« 

All  Windows: 


*default-label-character-style* 

Task  Stack  Display: 

*default-task-stack-character-style* 

Dependency  Graphs: 

default-task-node-character-style* 

Load  Plots: 


default-x-axis-character-style* 

default-y-axis-character-style* 

*default-axis-color*  3 
*default-axis-label-color* 
default-cumulative-load-line-color* 

Gantt  Schedule  Window: 

*default-pdg-Iabel-character-style* 

task-load-character-style* 

dw..*default-ruIer-normal-character-stvIe*  mM  fnr  iaK*r 

chart  siy,e  uscd  for  labeling  axis  margins  is  Gantt 

dw::*default-ruler-small-character-style* 

Files: 


Plot-lisp:  code  fo^gTierat^gSd  pSs  Vanables  common  to  several  display  types. 

Gantt,  lisp:  code  for  scheduling  Gantt  chart. 

pdg.lisp:  code  for  dependency  graphs. 

task-stack.lisp:  code  for  task  stack  display. 

pnnt-frame.lisp:  code  for  frame  displays 

TC'  PMK  fc-***r 


Page  B-40 


color.lisp:  variables  and  functions  for  displays  on  color  monitors. 

5.2.2  External  Interface  Detailed  Design 

Getting  into  the  details  of  the  interface  requirements  for  Z-Scheduler.  «e  have  outlined  them 
in  the  implementation-independent  BNF  form. 

The  following  are  Global  Variables  that  should  be  set  by  the  Task  Generator  during  Z- 
Scheduler's  invocation. 

Z invoker  ..  ,plavor  jnstance  that  has  a method  (like  "make-instance 
■.after"  method)  that  invokes  Z-Scheduler. 

♦primary-strategy*  ::=  MINIMIZE-TOTAL-TIME I BALANCE-LOAD 
p mary  By  . j^e  scheduling  strategy. 

♦time-available*  ::=  integer ; the  time  period  within  which  all  the 
time  avaiiduic  B ; input  tasks  need  to  be  performed. 

♦secondary-strategies*  ::=((<performance-measure><desirable-value>) ) 

• Each  element  is  a list  that  has  2 elements. 

The  first  is  an  evaluation  function  and  the 
second  is  the  desirable  value  for  it.  For  example, 

the  evaluation  function  may  be  c 

TOTAL-MOVEMENT-TIME-BETWEEN-TASKS  - sum  of 

movement  time  of  all  tasks  from  itself  to  its  ^ 

succeeding  task,  and  the  desirable  value  may  be 

’minimize. 

<performance-measure  ::=  ( <function-name> ) 

<desirable-value>  ::=  Integer  I minimize  I maximize 

*tasks-to-be-scheduled*  (<task-l>  <task-2>  ,...<task-j> <lask-n>) 

<task-j> #<mk-j>  _ m insMce  of  flavor  laskl. 

#<task-j>  must  have  the  following  accessors, 
name  ::=  symbol 

SS^uKdon  integer  ; jf** 

priority  ::=  integer  ; [1-10:  default  is  5] 

resource-required  (<V>  <C>  <A>  <M>) 

. r->  >1  . *La  /Mifrvnt  u;il 


constraints-by-Z  ::=  nil 


the  output  will  be  placed 
here  according  to  the  BNF 
format  specified  by  Jerry. 


#<pilot>  :=  Instance  of  a pilot  flavor. 
<V>,  <C>,  <A>,  <M>  ::=  integers 
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* input-constraint-list*  ( <singular-constraint-l> 

<singular-constraint-2> 

<singular-constraint-i> 

<singular-constraint-n> 

<disjunct-constraint- 1 > 

<disjunct-constraint-2> 

<disjunct-constraint-j> 

<disjunct-constraint-m>) 

; List  of  n # of  singular  constraints  and  m # of 
; disjunct  constraints  - where  n and  m are  variables. 

<singular-constramt-i>  ::=  (<type>  <reference-task> 

<relative-task/time>) 

<disjunct-constraint-j>  ::=  (OR  <singular-constraint- 1 > 

<singular-constraint-2> 


<singular-constraint-r>) 

<type>  ..=  Before  V After  V During  V Contains  V Overlaps  V 
Overlapped-by  V Meets  V Met-by  V Starts  V Ends  V 
Equals  V Starts-at  V Starts-by  V Ends-at  V Ends-by 

<reference-task>  ::=  #<task-a>  ; instance  of  task  that  is  a member 

; of  * tasks- to-be-scheduled* 

<related-task/time>  ::=  If  <type>  = Starts-at  V Starts-by  V Ends-at  V Ends-by 
Then  <related-task/time>  ::=  integer 
Else  <related-task/time>  ::=  #<task-b> 

Although  the  above  interface  principles  have  been  implemented  in  the  Z-Scheduler  its 
soundness  remains  to  be  tested. 

6.0  USER’S  GUIDE 

Please  refer  to  the  demo  manual  for  the  details  in  using  the  Z-Scheduler. 

6.1  OVERVIEW  OF  FILE  STRUCTURES 

directory^8  constra*n’n£  source  code  for  Z-Scheduler  is  neatly  organized  under  the 
Puf:>Renuka>Z>. 


Each  knowledge  source  has  its  own  directory.  Hence  the  above  directory  has  five  sub 
directories,  viz., 


>constraint-solver-ks> 
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>task-selector-ks> 

>Resource-allocator-ks> 

>constraint-propagator-ks> 

>truth-maintainer-ks> 


Each  knowledge  source  directory  contains 


the  corresponding  files. 


viz.. 


♦.rule 

♦.frame 

♦.log 

♦.lisp 


« 2 instructions  for  running  z-schedueer  demonstration 

The  following  instructions  ^ 

demonstration  is  perforated  on  Puffer,  a Symbolics  3641)  ut  tne 


Please  note  that: 

The  commands  to  be  typed  in  by  the  user  are  underlined 
The  prompt  provided  by  Symbolics  is  printed  in  italics. 
The  keys  that  are  to  be  pressed  are  outlined. 


6.2.1  Initial  Setup 

6.2.1.1  Halt  Machine 

TypCUtt  at  die  Command:  prompt.  The  machine  comes  ban,  * -he  «— 

^IX^"***'’**'****  I-sfyP'i^ory^'-.tmmm. 

6.2. 1.2.  Boot 

Type  EsStLat  the PEP  ^nt^o^eSe^isme'aM  that 

^SeTs1n“ m — P—  - - — 3 — 

6.2.1.3  Load  Z-Scheduler  files 

A,  the  Command:  prompt  type  UxiEkM  I t mfe ^an-inrrfacf  >be8in-UaLan 
return. 

At  the  Command:  prompt  type  linitial-selPfcaandhittMum. 

6 2.1.4  Run  Z-Scheduler  partially 
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bfcSSs.Mto  taST” ,hc  *ymbo1  md  0 **» 

pane  and  mouse  Tm™Sactionsm!  cSttSS  taton'A  ’Of**’*  up,othc  command  menu 
down  ,o  Stan,  Execution  and  dick  mouse Eft  Kn  A m'n“  a«,eaK'  ^ the  mouse 

Scheduler  nn  Tor  Hew  rnnutef  on  th'  color  monitor.  Ul  the  Z- 

KSr?o£^^ 

the  time  is  limited,  that  is  the  whole  demnncUf  * arts.of  ^ f '-schedul luig  activity.  Also,  since 

sassss »«SF^ttssss^ 

6.2.2  Input  Representation  Demo 
6.2.2. 1 Show  a task  frame 

window3  JlaSil^theup^r' Teftl  h^dkJmS)  This  Ic?™  i"ntl?c.,Unschcdu,ed  Tasks" 
looks  like  a small  right  angle  thatmoves  whhih^m  3?°n  WlU,b/lng  UP  a button-  which 
implies  that  the  window  that  displays  the  task  descrin^6™  tbc/no“se.  This  response 
this,  click  mouse  right  and  then,  holding  the  mouse  left  burned*0  **  ShaPed  1,1  order  to  do 
about  on  quarter  the  size  of  the  whole  color  screen  andH^wu  d°Wn’ outIlnc  a window 

This  action  leads  a display  of  the  details  of  the  Sos?n  task  ^^T  ^ butt0n  again' 
the  mouse  actions.  cnosen  task  in  the  window  just  created  with 

wi^drodwSCnbmg  ** t3Sk  S input* remove  the  tesk  wind°w  by  clicking  mouse-left  outside  the 
6.2.2.2  Show  a constraint  frame 

one •xSSuptomnS^tSk^^  rih *****  **“ this time click on 
nght  hand  comer).  Also  rememher  in  do^  (thc  window  place  on  the  upper 

in  length  and  breadth.  The  smaller  size  of  the' wiSo^i?rS^°H  ^d°W  only  3 **^5 
information  will  be  displayed  in  the  constrainMef^ril^ recommei nded  since  very  little 

~ft  —•  - “ 

6.2.3  Schedule  Generation  Demonstration 
6.2.3.1  Resume  Z-Scheduler 

JSitrssK  r ng  the  ^ °-  *. 

Th*S  acd°"  WUI  !pur  ■>"  Z-^'cc  <o  continue  .S”  l$££j  £ £££“■ 
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where  il  left  off  due  to  the  previous  suspend  action.  After  explaining  the  reheduiitig  process 
wX5  aid  of*the  dynamic  displays,  mouse  over  the  Suspend  menu  optton  and  cltck 

mouse-left  again. 

6.2.4.  Scheduling  Strategies  Comparison  Demonstration 

6.2.4.1  Bring  up  the  display 


7.0 


abbreviations  and  acronyms 


Army  Aircrew  Aircraft  Integration 

APU  — Auxiliary  Power  Unit 
BB  — Blackboard 

CSP  — Constraint  Satisfaction  Problem 
EST  — Earliest  Start  Time 
ET  — End  Time 
KS  — Knowledge  Source 

LET  — Latest  End  Time  . . „ 

MIDAS  — Man-machine  Interface  Design  Assistant 
PDG  — Partial  Dependency  Graph 
ST  — Start  Time 


8.0  NOTES 

8.1  MISCELLANEOUS 

SS5a»  foreqdpment  allocation. 

8.2  LIMITATIONS 

Although  the  interface  between  the  Task  Generator  and  the  Z-Scheduler  is  implemented,  it 
has  nofbeen  exercised  and  hence  its  efficiency  remains  unknown. 

8.3  LESSONS  LEARNED 

£ raJSi on .he  benefits  of , his  architecture. 

Murce'^^alMe^al^hof  thiTappcM^ttc^l^s^iti^thOTismOTtlykamalcmitrol 

5£ knowledge  sources  although  there  is  room  for  parallelism 
within  each  knowledge  source. 
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into  a 10  units  of  1 sec  each,  then  each  task  must  start  at  any  of  those  10  markines  and 
cannot  start  at  1.  5 sec  etc.  Hence,  considering  a time  line  of  discrete  intervals  constrains  the 
granularity  of  the  schedule  by  the  time  interval  of  the  smallest  chosen  interval.  Of  course, 
the  foremost  reason  behind  discretizing  the  time  line  is  for  putting  an  explicit  bound  on  the 
time  and  space  complexity  of  the  scheduling  algorithm.  P 6 

8.4  FUTURE  DIRECTIONS 

«SJHJnI?SnreiSearCh  WOrk’  tI!erLe  “*  always  numerous  ways  in  which  the  work  can  be 
extended.  Below  are  some  of  the  enhancements  that  can  make  Z-Scheduler  more  powerful. 

^Hierarchies.  Currently  Z-Scheduler  handles  tasks  that  are  the  leaves  of  the 

Sm!n^Chy' 11  d<flS  n°,i havc. 1 *e  knowledge  to  either  abstract  the  tasks  into  higher 
groupings  nor  can  it  handle  tasks  given  to  it  as  a hierarchy.  Z-Scheduler  can  neither 
CTeate  hierarchies  nor  can  it  handle  ready-made  hierarchies.  If  Z-Scheduler  were  to 
be  extended  to  schedule  task  hierarchies,  I foresee  an  impact  on  the  time  required  for 
c eduhng.,  i.e.,  this  extension  will  shorten  the  time  required  for  scheduling. 

0 Resource  Hierarchies:  Haying  an  abstraction  of  the  resources  might  lead  to  a 
more  sophisticated  and  cognitively  closer  resource  allocation  model.  Once  again 

for  scheduling)6  ^ °f  reSOurces  mgh[  reduce  **  deliberation  time  (time  required 

° Snendule  5!PainASSlJming  that  there  wil1  **  a dghter  interaction  between  the 
1th6  sc|}fduler  — or  the  scheduler  and  the  simulation  in  Phase  V of  this 

f„c°^ct  therf  wlU  be  a need  to  quickly  repair  the  existing  schedule  to  allow  to  new 
tasks  generated  m response  to  some  unexcepted  events. 

0 tea!  re!atlons  relating  temporal  constraints:  In  the  current  version  of  Z- 
»nJ  tT  T c.onftraint  .mechanism  can  only  handle  singular  temporal  constraints 
and  not  he  logical  relations  (ORs,  ANDs,  EXORs,  NOTs)  that  relate  the  singular 

reffiI?K^nStraintS'  JnvStC5  Wlth  Kevin  Corker  s definitions  of  goals  (logical 

S g°Js)and  Procedures  (temporal  relations  between  procures),  if 

mlhwU  er  WerlL°  handle  goals  and  procedures  uniformly,  its  constraint 
mechanism  must  be  extended  to  reason  over  logical  constraints  too. 

0 Learning  Techniques  in  the  Scheduler:  There  have  been  proven  learning 
techniques  in  the  literature  and  some  of  them  are  yet  to  be  tried  in  the  scheduling 
domain.  From  discussions  with  Sandy  Hart’s  group,  it  was  concluded  that 
investigating  in  this  area  too  might  lead  to  directions  that  will  lead  to  a closer 
cognitive  model  of  human  scheduling  behavior. 

Sophisticated  backtracking  mechanism:  In  its  present  version,  the  Z- 

e<i^eri!,SeS  c.hr?noog!cal  backtracking  (refer  to  section  5.2. 1.5)-  Simply  stated 

nHkSth°  P0!™  wherc  a decision  regarding  resource  allocation  was 

made,  finds  the  alternate  choices  available  and  picks  the  next  best  alternate  and 

ESSU  eS  from  that  point.  Instead,  if  we  were  to  have  a dependency 
directed  backtracking  — in  the  event  a conflict  occurs  — the  scheduler  can  backtrack 
directly  to  the  source  of  the  conflict  rather  than  just  blindly  backtracking  in  timef^ 
This  again  is  an  efficiency  measure  of  the  scheduler's  behavior. 
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uses: 


•mere  are  also  certain  desirable  extensions  to  GEST  - the  underlying  tool  that  Z-Scheduler 

° Time  nml  function  in  the 

the  controller  does  not  behave  as  expected. 

with  the  arcs  being  specie  rules  that  explain  why  a specific  object  was  ch^iged  *e 
way  it  did.  It  is  a useful  graphical  explanation  utility  which  is  lacking  in  GES 
hence  a candidate  for  future  extension  feature  in  GbS  l . 

O PtntnHnnc  for  viewing  th<*  conflict  set:  Before  the  conflict  resolution  stage  and  the 
SSS  rf^mles  it  would  be  useful  to  view  all  the  rules  that  match  the 

sources. 

• llsrr  (frnnfil  memory  rrcliim^onfuntnai^p^ste^^^ic^^^ahTOg31 

.specially  While 

running  large  applications  programs  in  GEST. 


9.0  APPENDIX  A 
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MIDAS  Phase  IV  - Z • Scheduler 
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APPENDIX  a - SPEECH  FOR  Z-SCHEDULER  PRESENTATION 


Z-Scheduler  attempts  to  capture  »“ 

model.  In  this  demonstration  I^Ube  showmg: i ^cation  of  Uic  minimize  time  strategy 
strategy  and  the  balance  load  *trat®^:f.  . :n  as  smau  a time  window  as  possible  by 

Sw’dSiS:  of  tome  trudts.  In  the  Ktentmre.  especudly  *» 
by  Sandy  Hart,  these  strategies  are  well  accepted. 

As  you  have  just  heard  from  the  pfanning 

Scheduler  is  a set  of  tasks  to  be  «*eduled  ^ togpredict  the  future  tasks  within  a 

component  of  the  operator  model  when* he  op^wmes^m p ^ to  come  up  ^ ^ 
specific  time  horizon  and  hands  over  teendot  ctjed^  ^ ^ #chedukd  bunched 

acceptable  order  to  perfo™  s _.  . contains  details  on  the  estimated  duration  for  the 
together  in  this  stack.  Each  task  fec^pftoa < Xr  tasks  on  the  stack,  the 
task,  the  priority-  the  relative  impwta  * . h visual.  Auditory,  Cogmtive  and 

SSnnS  £ SSSSS  ^ *01  * s^ngnex. 

Along  with  the  stack  of xectr fiffrfSettdazSdSSte  can  handle 

^'xample'' we  c“ 

<??i£ * i? sss s ,ons 

expressive  set  of  constraints  for  the  user  of  this  too  . 

Now  that  the  inputs  of  the  o^^aly^^pending^ onAe  Mtnre  of  input, 

before  that  the  scheduling  problem  nc^s  to  be  y P®  ^ scheduling  problems 

Ben  Arieh  in  his  PhD  dissertation  fiom  Puri*; Lafic  solutions  that 
can  be  classified  and  for  each  category  of  cl  tua,  7's  scheduling  problem  is 

Sly.  Following  .he  same  framewmk ^ ^mi ifiSe  J£kx  end  of 

0kS.«taiU  Ml  •»  «**  1-e  P«*Mn  and  hence  we 
wm^to  knowledge-based  techniques  for  the  answers. 

As  die  majority  of  works  in  ^(^SMSHra^w^S'a^btok^tod  in  this 
problem  as  a constratnt  satrsfacnonp  , ’ ^ ^ looai  solutions  are  found  in 

ISrrreSS^^S^utgt^Tve^e  c!p  T*.  techni„ue  wdl  * clear 
as  I get  into  the  scheduling  process  itself. 

Now  moving  to  die  Man  of f the ^uUngpoeess. “ 

23Sf  , \SS2 is^ ’a^SS^es  of  te  MU  genemnon.  The  knowledge 
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r“i  *■ 

into  a uniform  base  for  the  temporal  reasoni no^Aic«Ci^C  "y  r representational  constraints 

zimmmT- 

a'S  'S  “>  Pick 

w°  — 

TOsSCuiS  K;ttfxs^^J^c-t,ed  "?  measum. 

r*  T“.  “ °f  **  shck 

solution  that  has  minimum  resoi^XKhdtm  h?BSn.Sdk:ttdiis -° pick  ** 

«t  away  into  the  opportunity  of  the  tasks'**  haVjnitfS&S^  “°"  "°* 

te^SSbXffi » 5SSS.SE  !fi~ojl  keeping  knowledge  soume.  Eve* 
source  halts  the  execution J S ^ ^ ProP?8atin8-  *•*  knowledge 

SKSS 

hrfom  die  conflict  wSlS^odS  ^ W “ *“ 

KS,'^"0'0®031  b“klnKkta*'  *h're  *•  ^"i  taeteSfil^  the  last 


Annex  C 


Army-NASA  Aircrew/Aircraft  Integration  Program:  Phase  IV 


Man-Machine  Integration  Design  and  Analysis  System  (MIDAS) 
Cnffwarp  retailed  Design  Document 


Task  Loading  Model 


prepared  by 
Lowell  Staveland 


Table  of  Contents 


i.°  oFboCTJMEOT::::::::::: > 

li  SSSoSJSS58BSKi«*<»fiseeii^  j 

1 3.1  Purpose  of  Document 2 

1.3.2  Objectives  of  Document . . . . . . . . 2 

20  related  documentation..^. 2 

2.2  INFORMATION  DOCUMENTS 5 

3 0 CONCEPT •"•••"  •" 5 

3.1  DEFINITION  OF  TLM 6 

3 1.1  Purpose  and  Scope 6 

3.U.1  Purpose  of  TLM ....- ZZZZZZ 6 

3.1. 1.2  Scope  of  6 

3 1.2  Goals  and  Objectives <3 

3 1.2.1  Goals  of  TLM 6 

3 1 2.2  Objectives  of  TLM 7 

3 13  Description  of  TLM - 7 

3 1 3.1  Background - 7 

3 1 3.2  Definition  of  Loading 7 

3.1. 3.3  Approach  to  development.... ...8 

3. 1.3.4  conceptual  9 

3. 1.3.5  Dimensions  of  the  MIDAS  AS  TLM 12 

3. 1 3 .7  Definition  of  Resources  used  in  th  ........  1 2 

3 13  8 Rank  Order  of  Resource  Use 12 

3.13.8.1  Ranking  procedure 12 

3.1.3.8.2  Ranking  scale 12 

3 13  9 Generating  Conflict  Matnx  Values......... 

3 13  9 1 Generating  within-matnx  values....... 

3 3.1.3.9  U Example  1:  Two  tasks  with  two 

ST i9L2  ^m^2?Two  tasks  with  a 

near  visual  and  a far  visual  requirement J 

3 1. 3.9.2  Generating  between-matnx  values 

3 1.3.10  Classifying  the  PUtrt-Task  foteracOOTL.^. 14 

3 i 3 10  1 Cognitive  Task  Analysis  (CTA) 

MIDAS-TLM  as  a CTA  Structure 5 

3 13  10.3  Mapping  a Task  to  the  Taxonomy  

3.1.3.10.4  Taxonomic  Subsets  Used  to  Classify  ^ 

Pilot-Task  Interaction  17 

3 1.3.1 1 Calculating  the  Loading  Values 19 

3.2  USER  DEFINITION ..-•••-■ ’Xo  aV^RKTICS Z"Z1Z”ZZZ  20 

^3  CAPABILITIES  AND  CHARACTERISTICS 20 

3.3.1  Architecture 20 

3.3.2  Process  Capabilities 20 

3.3.3  Performance Z ........  • 20 

3 3 4 Interfaces 21 

3 35  Error  recovery  capabilities 21 

3 3.6  Reliability 21 

3 3.7  Maintainability 21 

3.3.8  Flexibility  and  Expansion ••••••” 21 

3.3.9  Transportability ZZZZ”. 21 

I f ll  Adaptation  to'  Various  Operational  Sites 21 


1 


Table  of  Contents 


3.3.12  Phased  Implementation 

t53'ur>Gen!ral  F1°W  °f  Data 55 

5H1  £eneral1 ^OW of Execution  Control “ 55 

« PKSggi'"'^ 

4.3  SOFTWARE  ENVIRONMENT  24 

4.4  externalinterface requdie'm^ots 51 

4.4.1  The  Designer’s  Interface....  51 

4.4  1.1  Purpose  of  the  Interface....'. 51 

4.4.  1 .2  Information  Content  of  Displays. . . .' 5c 

11' V3.  Intcracting  with  the  Interface 5c 

aao  tk'V' It  Interface  Constraints 5 1 

11*21  Purpose  of  the  Interface 5c 

4.4.2.2  Information  Content 

4.4.2.3  Information  Flow ~ 

5.0  DESIGN 4.  4 2 4 Implementation  Constraints 26 

5. 1 architectural’  design’.’’.'.’.'.'.'.' 26 

5.2  DETAnj2DE^^N!^!l^.^|^j^ 

5 2 2 FG2-  ThVflVt411^  a"d  daVa-input’fu’nc'do'nai'  group 27 

523  FG3  Calculator  functional  group™.  P 5n 

524  FG4-  TheS?  T,d°WS  3nd  menus  functmnal  group:.’ 34 

525  Fr/s  th?  !ask‘  oad  formatting  functional  group.  P A 

526  FP6-'  ZJje nndas-inteiface functional  group ...... . 15 

5 2 7 lei-  I?e  system-initialization  functional  group.... 2 

usERs2’7GmDE.  Um'system  functionaJ  group 

6.1  OVERVIEW  OF  PURPOSE  A ND  FT T isi r'TT r\ kt c 51 

6.2  INSTALLATION  AND  INmXzATinMIONS 51 

6.3  CTARTUPANDTERMNATON  Z TI0N 52 

^'1  ™£?'II9NS  AND  THEIR  OPERATION  

6.6  RECOVERY  STEPS . ...  S 


6.0 


..54 

...54 

...55 

...58 

...58 

...58 

...58 


8.0  ABOUDSSAR  YS  AND  ACRON™S 

9.1  LIMITATIONS.. 

9.2  LESSONS  LEARNED .' 

mo  a 2£35{rURE  SECTIONS . 

10.0  APPENDIX  A 

Firnpp  ACT^V1^trix  for  ^ Visual  Dimension 

FIGURE  A-6  CONFLICT  MATRIX  FOR  trf  DIMENSION 64 

houre  a-,  oonfu£» 


11 


Table  of  Contents 


Table  1.  Taxonomy  of  Dimensions  and  Paired  Elements  Classifying. 

Table  2.  The  Properties,  Structures  and  Processes,  and  tages 

Table  3.  Definitions  of  Paired  Elements 


10 

11 

16 


iii 


man-machine  integration  ^^^^Iument  TEM 

(MIDAS)  SOFTWARE  DETAILED  DESIGN  DULUMt.ni 
' PHASE  IV: 

TASK  LOADING  MODEL 


tware  Product  Specification  for  the  Task  LoadingModel  (TLM) 
line  Integration  Design  and  Analysis  System  (MIDAS). 


1.0  INTRODUCTION 

There  are  five  main  parts  to  this  document:  (1)  the  scope,  purpose  objectives  of  the 

^IDENTIFICATION  OF  DOCUMENT 

This  document  is  the  Software 
module  of  the  Man-machine 

1.2  SCOPE  OF  DOCUMENT 

Thic  rWiiment  describes  a computational  approach  to  modeling  human  information 

4 demands  Aa,  am  imposed  on  an  operaior  of  complex 

systems. 

within  the  MIDAS  workstation. 

toS^Sfed  into  the  model  and  applied  to  the  MID  AS  stmnlanon  environment, 
asnects  of  the  TLM  human-computer  interface  will  be  discussed  together  wJt^Vf 

contained  within  the  TLM. 

The  readers  of  this  document  should  have  knowledge  of  different  human 

modeling  techniques  or  human  information  processing  theories,  along  vn  ^ 

with  the  Symbolics  programming  environment  and  object-oriented  programming. 

1.3  PURPOSE  AND  OBJECTIVE  OF  DOCUMENT 

1.3.1  Purpose  of  Document 
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dSS!  al'en“,iV'  crews,a,i0" 

Svelo“™m"‘fKd'S  S°f,Ware  cn8inecrine  specifications  for  taint  TLM 

1.3.2  Objectives  of  Document 

configutanons  on  the  mfonnadon  processing  capabilities  of  a pilot  or  o^Tof  a S.plex 

The  material  in  this  document  is  directed  toward  three  categories  of  readers: 

1)  Those  who  wish  to  learn  what  the  MIDAS  TLM  accomplishes. 

2)  Those  who  wish  to  use  the  TLM  software  to  investigate  the  interactions  between  an 
operator  and  a specific  crew  station  design  within  dfe  coStffg”™  SSSK 

3)  Those  who  might  want  to  modify  and  update  the  TLM. 

2.0  RELATED  DOCUMENTATION 

2.1  APPLICABLE  DOCUMENTS 

The  following  documents  are  referenced  herein  and  are  directly  applicable  to  this  volume: 

| 

2.2  INFORMATION  DOCUMENTS 

The  following  documents  amplify  or  clarify  the  information  presented  in  this  volume: 
^^eevDnEf^faselM°S^^CR°^r^°^^^a^^^®  s^te^de^^L^CLMacMillan, 

££^5^  h"™° 
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Boff,K.R.  & Lincoln,  J.E. 

information  processing,  in 

Thomas  (Eds.),  Handbook  ana sons 

rr~* ra  »nH  performance-  (PP-  28-1  - 28-71).  JNew  ior 

Derrick,  W.  (1988).  Dimensions  of  operator  workload.  Human  Facias,  2Q,  95-110. 
Dunn-Rankin,  P.  (1983).  Scaling  metbflds,  Hillsdale,  New  Jersey:  Lawrence  Erlbaum 
Associates. 

Sg^S!g!Se^CS:,'* 

Fleishman  E.  A.  & Quaintance.  M.  K.  (1984),lteliamte  of  MmP  I**"1™1**  The 
SSi  hnmantasks.  Orlando:  Academtc  Press. 

Gibson,  J.  I.  (1979).  The  emlnaical  aonroach  10  visiwl  PCTOiman  Boston.  Houghton 
Mifflin. 

Gopher,  D.  & Kimchi,  R.  (1989).  Engineering  psychology.  AppH  Review  of 
Psychology,  4&  431-55. 

,.  c nnsA'i  Workload  - An  examination  of  the  concept  In  K R. 

KaS“  R Thom-  (E«:Sf5^MTf^&ew  Yorh: 
rrf;rm,nre:  vo<  H Pognitive  proc^s^  and  performaosg-  1PP- 

John  Wiley  and  Sons. 

perfo^^ 

(pp.  ).  Colombus,  Ohio. 

Halt, S. G.  & Staveland, L. E. (1988). ^tlo£™"A°H«o*liN . MeshkMUBis.),  ' 
l^oloadtS?^!  MO.  Amsterdam,  The  Netherlands:  North  Holland. 

Holley,  C.  D.  (1989)  A mode,  (gx** 

Plenum  Press. 

Hulme,  A.  J.  & Hamilton,  W.  I.  (1989),  Human  engin, Knng  modelstA  user^^  ^ 

raddi  ■" W tain  (pp-  “”-**»■ 

York:  Plenum  Press. 

Hall 
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Lachman,  R.,  Lachman,  J.  & Butterfield  E C rtQ7o\  n 

pro^inr  An  intrmlm 

Operator  woiWoadfor^Ut^y^y^^*l^u-^i  Blttf cp. A c-  & Christ,  R.  E.  (1989) 

M.  Strub,  R.  Sutton  & L.  V^BiSa  (Eds9)  An^ra?  ° AJtcMiUan> D Bcc™,  E.  Salas, 
^LSYStcm design  (pp. 21-46). New ^c-pjSfpSS °f hll™n ^orman^ model, 

McCracken,  J.  H.  &,  Aldrich  T R a i 


Miller,  ^ J a sacin  ski  t ^toj2Q\  ru 

sample^ ; control  skills  (Final  Rejirt  Gra^0gNAu TySs?  v7ffPtiffl  nrrion  in 

Research  Center,  NASA  (RF  PrJ^ct  7^264/7 14826) . ' Moffctt  F,dd*  CA:  Ame- 

\T  . 1 n 


MacMPla^D.Beevis  E&lteM  Smbc”'?  ’"“’r  °f °Pcr>tor workload.  InO 

O nnnnflll  D "r\  o . 


John  Wiley  and  Sons 

research.’  £‘£'ffowto^  time  in. information  processing 

ad  recognition.  Pmom*.  (Md 7fiS|™^nf™non  processing-  Tmorhis  in  ^J^mrr 

in  fomiahonS  processes5 ' In  P?  A .^Hancock  ^ M^Chf'  \ 2£89)-  Human  interactive 

iSSi^  (pp- 129-]64)-  fflHS?eKe 

Handbook  of  JZgtaazkS^huma^  n^',Boff’  L'i  ^flj1f”an,  *.J-  p-  Thomas  (Eds.), 

(pp-  v-3  - wigS^gf  w ff^PgYr  prorrm  nnd 

other distinctions  in  human 'perfoma^e  r3els S!*nS’  and0symbols.  and 
Cybernetics.  12(3),  257-266.  ' Transactions  on  Systems  Man  )1nf| 

Roth,  E.  M.  Sl  Woods  D n noso\  o • . 

acquisition  for  intelligent'system  design 1 GGuidaTr8^ A"  ?£5rt?ch  to  knowledge 
systems  design,  (pp.  ).  Amsterdam:  North  Holland.  C‘  T s°  (Eds  )’  Tomes 

D.  Beevis,  E.  S ias  JV1 . ^Va^  BredaTErf"!  fe  G MacMUlan, 

performance  mixlrh  to  system  design-  (pp.  475 -486)  New Yoi£- hSS0™ 

Schneider,  W.  & Fisk  A D a** 

psrfomiances  (Rep.  hapi  '^np  ^^cI^ .’iTJT  ?n<j  mfrhf,ni!ms  for skilled 
Attention  Research  Laboratory.  ' Ch  p gn'  University  of  Illinois,  Human 


Page  C-4 


Staveland,  L.E.  (1988).  r™,hin„nml  roles i for  Iff nfffllinf  workload  ratrnm  Unpublished 
master's  thesis.  San  Jose  State  University,  San  Jose,  CA. 


master’s  thesis,  San  Jose  State  University,  San  Jose,  CA. 

Treisman,  Anne.  (1986).  Properties,  parts  ^ 

Uniuersdty  of MnKnginUrtog^ iSS  Uborntory. 

....  , p I-,  MQjMi  Processing  resources  in  attention.  In  R.  Parasuraman  & D.  R. 

Wickens,  C.  D.  Uy»4J.  rrwcssing  Orlando  FL*  Academic  Press. 

Davies  (Eds.),  Yar*^  of  attention  (pp.  63-102).  Urianao,  ru  ™,«uc 

Wickens  C.  D.  (1987).  Attention  in  Aviation.  PrPCftgdingS  Of  ttlf  4th  ConfgrglKg  Oil 
ftviation’Ps'vchology.  Columbus:  Ohio  State  University. 

....  . „c  p r»  riQROat  Models  of  multitask  situations.  In  G.  MacMillan,  D-Beevis,  E. 
Sas  M So?b,RSuton&L.  Van  Breda  (Eds.),  Applications  of  human  pgrforman££ 
Sifcu Sm  design,  (pp.  259-274).  New  York:  Plenum  Press. 


....  . c p n nQ89bl  Resource  management  and  time  sharing.  In  J.  Elkind,S.  Card, 

J Hochbere&B  Huey  (Eds  ),  Human  pgrfnrTrtance  models  for  COlTipUtgr  aided 
180  202).  Wa?hinpoi:  d'^ ^crNational  Acndemy  ness. 


b „ rn  * Andre  A D (1989)  PAWES  multiple  rt*<tf>nrce  analysis.  (Report  No. 
AFDayton  W-59630X).  Urbana-Champaign:  University  of  Illinois,  Aviation  Research 

Lab. 

Wickens,  C.  D.  & Flach,  J.  M.  (1988).  Information  Processing.  In  E.  Weiner  (Ed.), 
Human  factors  in  aviation,  (pp.  111-155). 

Woods  D D.  & Hollnagel,  E.  (1987).  Mapping  cognitive  demands  in  complex  problem- 
solving  worlds,  international  Journal  of  Man-Maching  Studio.  26, 257 

Woods,  D.  D.  & Roth,  E.  M.  (1988)  Cognitive  systems  “f 

(Ed.),  Handbook  of  humanTntriPntftr  interaction,  (pp.  3-43).  Amsterdam.  Else 

Science. 

3.0  CONCEPT 

This  section  provides  the  purpose,  scope,  objectives  and  details  of  the  TLM. 

3.1  DEFINITION  OF  TLM 
MIDAS  loading  model  isan 

model.  It  is  an  output  model  becaus  8 ^ aircrews  or  system  operators  are 

“jj^ 
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well  as  performed  serially.  8 ety  ot  tasks  Perfumed  concurrently  as 

Throughout  the  presentation  of  the  concept  the  term  l_,i  . 

3.1.1  Purpose  and  Scope 

3.1. 1.1  Purpose  of  TLM 

3.1. 1.2  Scope  of  TLM 

ssaasarassssffise 

SgSSriSSs. 

3.1.2  Goals  and  Objectives 

3.1.2.1  Goals  of  TLM 

aSS-HfSS—SSr-.. 

3.1.2.2  Objectives  of  TLM 

design^ 
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information  processes  and  s^ct^b  tgfUnderl5 ii^ws? performance.  Memory, 
perceptual,  cognitive  and  motor  between  aftributesin  the  taxonomy  are  compared 

attention,  and  time  demands  an^  nfl  c ^ accolding  to  a model  of  workload  that 

v^^l,  auditory,  cognitive,  and  motor  dimensions. 

3.1.3  Description  of  TLM 

of  the  model. 

3.1.3. 1 Background 

Hie  model  of  loading  previously  used  ^^^^^“Sj'jbec^fmultipte  resources 
McCracken  & Aldrich  (1984).  It  is  basaion  cSnidve!  and  Psychomotor  (VACP) 
that  partitions  loading  into  VtaA  Audru  ry,  C g jlot  ^uoad,  which  is 

a pilot  dunnPg  night"  (Aldrich.  Srabo  & 

Bierbaum,  1987). 

This  model  does  not  meet  the  need^ofthe 

are  assigned  by  the  design  team  g ^ interact  with  task  performance.  It  is 

This  reduces  the  model's  sensniviytothecontex^  mat  rformancc  depends  on  the 
important  to  incorporate  contextual  effects  be  jj  Qther  tasks>  which  can  have  a 

environment  in  which  it  is 5 perform >ed^  J without  accounting  for  context, 

The  MIDAS-TLM  will  enhance ; asks  are_8 
during  a simulation,  providing x ens it  grt  the  ^ ^ a msk  to  its  load  value.  These 
embedded,  and  by  anchoring  taste©  a tune  generating  predictive  estimates  of 

told  sg  - cmai”  wor,d'  “ 

environmental  airtbufes  that  arc  generated  dunng  a sunulahon. 

3 13  2 Definition  of  Loading 

demands. 

3.1.3. 3 Approach  to  development 

The  developmental  approach  is  SnteSthe  mental 

ar6Si^ 

underlying  the  MIDAS-TLM: 

• An  abstraction  hierarchy  comn^^^^fot^i^h^rdcc^i11!  . 

<snuctu.es)  “d  ma"ip,,,a,e  (PT0CBS'S)  inf0rma,'°n 


Page  C-7 


3.1.3.4 


* 119791  «“**>■  "I*  the  essence 

' j^fiSSssg- 

SiSSS5=SH«” 

(Dunn-RanJdn?  1989I  wSens 

Conceptual  Structure  of  the  MIDAS-TLM 

tffihng iSSSJSdlSSf » r Ww"ks  *^1*  fte  objective  aspects 

& Wickens  cSSS^  * 1 &**«£  WaSS 

Christ,  1989;  Sanders,  1989;  Hulme  & h2S£, Dick* Bittner  & 
frameworks  were  selected  as  a means  rnH^n^  ’ 989;  Wiclkens-  19g8)-  These  two 
or  more  tasks  and  are  necessary  to  provide  cornet' lnleractions  of  one 
interactions.  These  frameworks  inmrmmtP  °"text  sensitivity  and  to  assign  values  to  task 

.ha,  are  bon-owed  tom  Gibson's  ecohgcal  PfOP",ieS 

perceptual,  cognitive  and  motor  svstems  that  «~t«°  f 'nL  on  Present  in  the 

“*  the  perceptual,  cognitive  or  motor  activities  thaThTve^n.^  °f  .relations;  Processes 
(Lachman  et  al,  1979).  es  tnat  have  structures  as  inputs  or  outputs 

Serems^uTtoeT^nd ^SsSSomanVn  elernents ^?b,e  U represent 
invariant  properties.  These  dimensions  and  elemi>ifCeSSink ' 311(1  '"corporate  tiie  notion  of 
propenies  ofTnderiying  stiZSS prees^K  !)"“  ‘he  inV3,i3", 

’S?,™1™  PtopntiiK  .o  categorize  die 
structures  and  processes  £d  ?heSmcdo„?  Pr0p4",eS  WiIh  ® the 

scale  of  0 to  4.  moiy’  attentlona|.  and  temporal  resources  on  a relative 

3.1.3.5  Dimensions  of  the  MIDAS-TLM 

invari^t  properties  32^  ^ the 

properties  result  from  research  on  mental  wortinnH  .h*.  ’ ^ese  ^'mensions  and  their 
multidimensional  in  nature  Loadin8 is 

has  been  measured  in  many  different  ways  by  many  differed res^chmV!L80;iWn8& 


Page  C-8 


Eggemeier,  1986  for  a thorough  but  must 

l^S^comLed  a,  ^ dteorenca, 

level  to  derive  a loading  value  (Demck,  1988)* 

nylons  used  tuBiMg; AH-64 
capacity  of  the  system  to  attend  to  processing  demand  . 

•n*  McCracken  & Aldrich  workload  assessment  technique  *' 

Visual,  Auditory,  Cognitive  and  ^yohomotor^  , ^aie,  ,0  measure  pilot 

on  a ^ durine  fl,8ht 

(Aldrich,  Szabo  & Bierbaum,  1987). 

The MIDAS-TLM incorporates  essentially  jhe^ame ^atsresources  as  the 
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Scan  / Fixate 
Integral  / Separable 
Objects /Features 
Salient  / Masked 
Static /Dynamic 


“Auditory”"  Orient  / biscnminaic 
Signal  / Speech 
Salient  / Masked 


Spatial  Area 
Spatial  Search 
Spatial  Proximity 
Spatial  Grouping 
Spatial  Discriminability 
Motion  Cues 


Auditory  Localization  Process 
Auditory  codes  Structure 

Auditory  Discriminability  Structure 


tructure  Perceptual 

Process  input 

Structure 
Structure 
Structure 
Structure 


Process  I Perceptual 


Direct  / Transformation"  Automatic  I Control 

Processing 

Single  Choice  / Multiple  Levels  of  Processing 
Choice 

Verbal  / Spatial  Processing  Codes 

Planned  / Unplanned  Priming 


Motor 


"Verbal  / Spatial  Response  Loaes 

Near  /Far  Response  Proximity 

Discrete  / Continuous  Control  Dynamics 
Gross  /Fine  Control  Dynamics 


Process  Cognitive 
Processing 

Process 

Structure 

Process 


Structure  Motor 

Structure  Output 

Process 
Process 


Table  1.  The  Properties,  Sir “c‘“r“d™ Jiu'Snia “disused  in  the 

corresponding  to  the  d.mens.ons^djmmdae 

3.1.3.6  Invariant  properties  of  the  dimensions 

To  incorporate  the  multidimensional JnwnhOu ^'S'hrSimtpropenrM.'  tenvanant 
elements  are  grouped  into  dtmenoonsie  yjj  Kantotviu  4t  Roediger  (1980)  define  as 
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four  dmensions'wsuaJ?  AudfcSy  “Id  °“lp“l  ~ *> ■***  “oh  of  (he 

auditor  dimensions  visual  and 

cognitive  dimension  corresponds  to  the  Drocessfn m mp.u.t  modalltres.  The 
processing  information  to  various  depths  The  niiS  because  cognition  involves 
stage  because  outputs  are  generated  with  various  motofSfiStore.C°ITeSPOndS  l°  **  °UtpUt 


3.1.3.7 


Definition  of  Resources  used  in  the  MIDAS-TLM 


m m-iiva. 

demands,  or  the  amounT oTti^Sc^^ct^e  o^Dr^^^’-*6  am°Unt  of  attenti<>nal 
All  three  types  of  resources  « cSSdSf T\ L l?™?*  re^uires  «*>  Process  infonnation. 
Gibsonian  sense  (Glb^  wSSSSS * ^ w,!«P«*w  structures  in  the 
infonnation  extracted  from die mSSlS^  T <?' *e chanSe in *6 

the  variant  properties  of  the  structures  processes  these  resources  represent 

dimensions  in  the  taxonomy.  ’ P °Cesses’ and  sta8es  comprising  the  elements  and 

3.1.3.8  Rank  Order  of  Resource  Use 

re<K^  “jj  tiine  assumes  that  the  resource 

resource  requirements  of  the  other  element*  * * "°r(kred  V1  relation  to  the  implicit 

3.1. 3.8.1  Ranking  procedure 

Matrices  are  consuS^  with  itself  and  with  all  others. 

elements  between  dimensions  (betweemmatrixfThk^h^"/^^'™30"^ and  fora11 
four  between-matrices.  matnx)'  This  resuIts  ln  four  within-matrices  and 

element^  wlffSS expll^  and  contrasting  the  paired 

each  element  in  the  pair  and  relative  to  S rtSS. ? ^ “"P1*01  resource  requirements  of 
one  pair  of  elements  comparSwitih  °*W f™'  ™s  ™eans  for 

that  are  rank-ordered.  * another  P4*-  ^ are  four  interactions 

3. 1.3. 8. 2 Ranking  scale 

2SJSSS  « » * ' » ' -y  fcr  msoumc,  am 

greatest  amount  of  resources  are  required  The  rankin^s^C^ri^tf  SP^^i  ^ 4 11164115  the 
lnteracuons  were  treated  as  inviolable  and  9CC;™  ^ ^ 5t?cdy  ordinal.  Some 

shouldn't  be  used  to  calculate  loading Values ^ For  examSe0^85-3  flag  thatuthe  tateraction 
near  visual  field  and  at  an  object  in  the  far  vfsual  fielH  focus“S  at  411  object  in  the 
inviolable.  J VISuaJ  field  at  ^ same  time  is  considered  to  be 

3.1.3.9  Generating  Conflict  Matrix  Values 

dimension  are  the  Pairs  of  elements  in  the  visual 

point  of  intersection  matrix  (figure  1).  The 

element  in  a row  with  an  eleme^^ 
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two  pairs  at  a time  until  ‘f’’ SSteSdoK  geSStrf  W 
^^h^romto^adons  oTth/four  demenu  within  the  wo  pairs- 


viaurti- 

dimension 


["FA  I Sc  I H IbEIlN  I Ob 


Figurel.  OB=object, 

MA=masked,  ST=static,  DY=dynamic). 

3. 1.3.9. 1 Generating  within-matrix  values 

The  procedure  for  generating  the 

diagonal.  . 

For  example,  the  first  pau-  of  elements  (neat ^^JjeSn^  near 

generating  four  possible  interactions  diat  1*ffar  yisual  However,  since  the  near 
visual/far  visual,  far  visuaVne^  visual,  i jr.nl^(ions  ^ redundant,  only  three  interactions 

^«hSpaT^ 

UMU  Example  It  Two  .asks  wi.h  .wo  near  visual  requiremen.s 

Two  tasks  with  two  nearvisual  ^“f^^^'wSSTarvS^uiremMB.  The 
elements)  requires  fewer  resources  time  to  conduct  a visual  search  near 

time  demands  are  less  because 11 deeded  because  with  less  time  there  are 
PaWyfewe^sTain^toTs^d  in  short  ternt  memo*  - -coded  into  long  tern, 
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memory.  The  attentional 
though  the  same  amount 


CSS  beC3USe  m needed  fOT  less 
may  oe  required  for  a given  unit  of  time. 


time,  even 


3.1.3.9.1.2  Example  2: 
requirement 


Two  tasks  with  a near  visual  and  a far  visual 


resources  because  it  is  hard  to  divide  atientioSbSISen th^  prol^  f,y  n^ulre  the  most 
will  probably  increase  the  visual  near  and  far  visual  fields.  This 

and  forth  between  the  near  and  far  fields  t v!f  3 St?teg?  of  SWItching  attention  back 

than  focusing  on  only  one  or  the  otheifield  This  sraS  Sw  ^ more  time 

requirements  because  information  from  P^^bly  has  more  memory 

switch  in  attention.  UUOrTnatK>n  from  each  field  has  to  be  stored  and  recalled  with  each 

3.1.3.9.2  Generating  between-matrix  values 

elements  in  the  other  dimensions  This  reMtfS 5“  one  d“ncnsion  with  each  pair  of 
matrices  (four  within  and  four  KnEI?  ^procedures  results  in  eight  conflict 

Figure  1 (see  Appendk  A^F^urw  A-*l  throu^Cy^)L  & S,milar  ^as*,i°n  to  *he  matrix  in 

wmbiStioi^ofTe' fcJJSSSoS?  S^elfomXrthatWOUld  rcSUlt  6001 M 

serially  from  the  percemSS«t  18  assu.med  to  >»  processed 

(Wickens  &Flach,  1988).  Therefore8the  audfro™  prPCeS?,1,l?  .to  motor  outPut  stages 
input  stage  (1  matrix),  both  interact  with  thelSriS?  I?  modalltles  interact  within  the 
the  cognitive  processing  stage  interacts  with  thi^n-f  pJX)Cessin8  st?6e  (2  matrices),  and 
possible  interactions  within  the  eipht  matrix  i °r  S'a^e  ^ matrix).  Rank  ordering  all 
be  used  to  generate  the  VACM  loading values”*11  tS 310110(1 500  conflict  vaJues  that  can 

processes  underiy^  amon^th™^  that  the  structures  and 

rank  that  matched  experimental  results  testing  eitherThTr  ’ Each.interaction  was  assigned  a 
In  the  cases  where  interactionTcould^ ££ ™ ora  similar  interlction. 

extrapolated  from  the  most  appropriate  research  Bnfr  j£fCffiC  exP^iynents’  things  were 
Boff  & Lincoln  (1988)  were SSSSSSSf?'  ? ■’  Pufman  & Thomas  (1986)  and 
generate  the  rankings.  y ° °^>tain  x^c  experimental  results  used  to 

3.1.3.10  Classifying  the  Pilot-Task  Interaction 

3.1.3.10.1  Cognitive  Task  Analysis  (CTA) 

to  '^P^entadon  U«  pilot-usk 

states  of  the  world  (tasks)  the  state  of  the ^ LSS3S  ?\  ?Tg  the  currcn* 80(1  desired 

(W°°dS  & ROth’ 1988)  Whi™ **  exXaSShe°refore 

JganTS 
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solving  in  that  domain"  denSSstoat  are  transportable 

nrovides  a framework  with  which  to  captur  gn  ^ are  represented  in  the 

1988;  rX&  woais' 

1988). 

3.1.3.10.2  MIDAS-TLM  as  a CTA  Structure 


it***-*—  -- 

The  elements  and  dimensions  411(1  motor  demands 

structure  of  domain  semantics  that  organizes  me  f*P  & define  the  representations 

to  the  mks  that  are  available  to  the  pilot,  the  MIDAS-TLM 

ntioltrcic 


in  tne  pnoi-ias*.  

of  the  structures  and  processes  of  the - . 

taxonomy  can  guide  the  cognitive  task  analysis. 

3.1.3.10.3  Mapping  a Task  to  the  Taxonomy 

Task  attributes  are  mapped  to of Si^SSgthe principles  in 

stB&S&as±sxaes&^ 


structures  and  processes  ,o  Mch  pair 

of  dements  and  their  respective  dimensions. 

fingers)  and  their  combinations  (nght,  lett,  botnj. 

3., .3.10.4  Taxonomic  Subsets  Used  to  Ciassify  Pilot-Tasb  Interaction 

Not  every  element  of  the  ““^^^ts^ge'Ste  smams  llrfpSeSs  that  a 
interaction  because  the  taxonomy  represents  a rang  d processes.  Therefore, 

pilot  might  use,  and  different  tasks  re0utm  d.ffe rent  s^tums^p^  ^ pil0, .Bsk 

only  a subset  of  the  e,e™"K '"^fe possibte  subsets  that  can  characterize  the 

equipment  states  are  mapped  to  the  taxonomy. 

The  subset  characterizing  the 

Sice  they  are  structured  as  two distinct  t ^ ^ same  time,  both  can  be  absent, 
present  or  absent  and,  while  twenty  elements,  but  not  more.  This 
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Dimension  I ikiements 


Audjtory  Tirient/ Discriminate" 
Signal  / Speech 
Salient  / Masked 

Visual  /ParVisui 
Scan  / Fixate 
Integra] /Separable 
Objects  / Features 
Salient  / Masked 
Static  / Dynamic 


tone?51'"1'  C°IIlm;  SenSOry:  ^terpret  words,  ' 

Subsystem:  Comm;  Object:  messages,  alarms 
SednSStatC  variables:Jam^g.  high 

■irsuKystem  = internal  visUaJ  field  then  neaf 

If  sensoiy=(vb-list:orient)  then  scan; 
else  if  (vl.read) 

CDE-.  object  defs.;  World  model:  external  object 
CDE:  object  defs.;  World  model:  external  object 
Worid  model  state  variables:  night/day,  rain/clear 
Aero  model  state  variables:  stationary,  in  transit 


Snigle  Choice /Multiple  If  elenS,  (list.ditechflxate.sep,  then  choice 
Choice 

f element-(hst:trans,  scan-sep)  then  m-choice 
Verbal  /Spatial  t‘* 


Spatial  dse^SDada]read  (digits*words)  ^n  verbal 

Planned  / Unplanned  Scheduler,  (Planner):  inlOTUpred  mk 


verbal  /Spatial  ^ it  subsystem  = com  then  verbal  else  spatial' 

ear  / Far  If  jack-reach  < ?-distance  then  near  else  far 

Discrete/  Continuous  If  control . (object  list,  to  <Smlc 
nrrxoa  / r?*  e^se  continuous 

_ 1116 If  control  = (type  list)  then  gross  else  fine 


Ta  ill6  termcenflti0ns  °f  P?ired  Elements 

(examples  of  state  variables  the  AH-64  task 
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^SSSSftj«^ 

The  algorithm  calculates  a loading  value  for  each  dimension.  Ly.  A. 

ranks  of  the  paired  elements  in  the  conflict  matrices  are  used  to  calculate  these  valu  . 

Tho  subre,  of  elements  that  are  selected  via  iff 

the  elements  in  the  matrices  whose  in“^Q^d^JCcSSSix  and  used  to  calculate 

die  loadrng .values  to  between-  and  within-dimension- 

auditory,  cognitive,  and  motor  dimension. 

When  there  is  only  one  task,  s-1 , the  equation  for  the  withinKiimension-matrix  values, 

Cw,  is  given  by 

s n-2  n-1 

Cw  = X X.  . X , 

t=l  i=l  k=i+l 

atH  atii  atiic  = individual  conflict  rankings 
i,  j,  k = indices  to  the  columns  and  rows  in  the  matrices 
s = number  of  concurrent  tasks 
n = number  rows  and  columns 
t = task  index 

When  there  is  more  than  one  task  performed  concurrendy,  s>l,  the  equation  for  the  within- 
dimension -matrix  values,  Cy/,  is  given  y 


J+1  ptkAji^^ji^] 


Cw=,Si 


n-1  n-1 

I I 


n 

I 


L i=l  k=l  j=k+l 


^atkiJ 


tji 


>♦£  1, , i,w. 


atwi,  at;;  atik  = individual  conflict  rankings 

i,  j,  k = indices  to  the  columns  and  rows  in  the  matrices 
s = number  of  concurrent  tasks 
n = number  rows  and  columns 
t = task  index 
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dimension-matrix  vaiu^s'cB^  g^/by  S~'  ors>1’ the  equation  for  the  within- 


CB  = I 
t=l 


-k?i  i?i  li  j+>y] 


atki>  atmi,  atki  = individual  conflict  rankings 

j’  *■’  m®  ln“ces  to  ^e  columns  and  rows  in  the  matrices 
s = number  of  concurrent  tasks  mamces 

n = number  of  rows 
1 = number  of  columns 
t = task  index 

TTte  load  value  for  the  Visual  dimension  is  given  by 

LV  = Vcw  + [fVAbw  + VCbw)/2J 

Vcw  = loading  value  for  the  visual  within-matrix 
VA5w  ==  loading  values  for  the  visual-auditory  between-matrix 
bw  - loading  values  for  the  visual-cognitive  between-matrix 

The  load  value  for  the  Auditory  dimension  is  given  by 

LA  = Acw  + [( VAbw  + ACbw)  / 2] 

v aW  = *oading  value  f°r  ,he  auditory  within-matrix 

VAbw  - loading  values  for  the  visual-auditory  between-matrix 

ACbw  - loading  values  for  the  auditoty-cogmtive  between-matrix 

The  load  value  for  the  Cognitive  dimension  is  given  by 

LC  = Ccw  + [<ACbw  + VCbw  + CMbw)  / 3] 

Ccw  = loading  value  for  the  cognitive  within-matrix 

ACbw  - loading  values  for  the  auditory-auditory  between-matrix 

CM  = , Jng  ,UeS  f°r  the  ^“al-cognitive  between-matrix 
CMbw  _ loading  values  for  the  cognitive-motor  between-matrix 

The  load  value  for  the  Motor  dimension  is  given  by 

LM  = MCvv  + CMbA 

Mew  = loading  value  for  the  motor  within-matrix 

CMbw  - loading  values  for  the  cognitive motor  between-matrix 

(Wickens  & Andre.  1989)  - coss-mnUiplyi  load 
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values  than  cross-multiplying  low  r^s-  xv^eJt£reareSn5*  tSks^s  = 1),  each 

multiplications  are  summed  wU^,n  ®ach!J^s  JJat  are  cross-multipUed  across  rows  and 
conflict  matrixis  ^me=al  = of  ^ ^ foPur  between  matnx 

When  there  are  combined  bsMs  = the  ^^^^^mfft^tSnted  as  a 
multiplying,  in  a pairwise  prooodm e meeting  se  al  ^ rows  columns  in  the 

SXr,re^  ^ C'Bh‘  V*“  “ 

*Wch  an  avereging  and  summing  algonthm  ts  applted. 

This  averaging  and  summing  algorithm  is  basrf  on ‘ wWle  the 

averaging  models  under-esnmate  loads  and  add^e  monels  ove  (Staveland, 

Mug"  model  meettng 

The  serial  constraints  used  of 

between  matrices  are  detenmnd Jj  . SSis  (the  input  modalities)  interact  with 

the  information.  The  visual  and  audi  oiy  dimensio  ( v the  information,  and 

central  processing  dimension  central  processing 

SSeS’ motor  dimension)  since  it  determines  and 

monitors  the  course  of  action. 

The  final  load  value  for  each  di!’’Cn*1?”  “(^OT^ng^wagedvtdK.  Adding  the  value 
g£ SfiESSR  “«•  betwecn-matrix  values  represen.  the  "add., tonal 

cost"  in  the  "best-fitting"  model. 

. a ifvaH  value  for  the  auditory  dimension  is  calculated  by  averaging 

For  example,  the  averaged Toad  value  for  the  “ vaiUes.  Serial  processing 

the  auditory-visual  and  auditory-cog  . -.u  ,ue  cognitive  and  visual  dimensions 

constrains  ihe  auditory  dimension  is  computed  by 

load  value. 


The  resultant  load  values 11141  ^ *5^ 

aiea  _ J transform.  then  multiplied 


of  the 

The  resultant  iuau  vatuu, ... ~ ~-\F~  vfL  ,hp  number  of  interactions  tnat  are  allowed. 

number  of  multiplications  associated  wit  transform,  then  multiplied  by  ten  to 

Consequently,  die  load  this  model  should  be  able  to 

» - an,  number  of  interacting  tashs. 

3,2  USER  DEFINITION 

A user  is  defined  as  either  a member  of  the  design  team,  or  as  a MIDAS  component  that 
interfaces  with  the  MIDAS  TLM. 

Members  of  the  design  team  and  human  factors 

Sy?." 8 rS"«esi|n  team  may  be'required  to  udlize  the  tools  of  the 
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output  of  th^MsufficiSy  to  ju^th^fty  S cffic^  befabte 10  Undcrstand  ** 
TLM  structures  the  output  to  achieve  this  goal  vet  akn  nmC  °f  ®^°?cePtual  design.  The 
allow  for  a more  detailed  analysis  of  a concemiiai  rw°  pr°vides  sufficient  information  to 
and  human  factoring  of  desien  In  nn-w  csign  by  specialists  in  the  psychology 

design  team  will  neffiSf.  bacS  1“  'Sl^  “T' in  fi,U- a ™"’*r  oiX 

Scheduler  Md  ^a^pre£rad?„"H^  Uk  TLM:  lhe 

future,  a planning  model  would  also  u&Tthe  TTM^Th? crJyT^°1C  ®Pe™tor  Model.  In  the 
run-time  of  a simulation  to  obtain  the &d  • Jake,  h?d-ICr,CaIls  Ac  ^ duri*g 

^tegy.  jask  representation  methods  currentlySll  the  TLM^f16"1 8 load,b^lancing 
obtain  the  load  va  ues  necessary  to  seanenr^  t^iT  before  a simulation  to 

call  the  TLM,  the  scheduhng^mSt  ?s^ onl ‘°,be  simuIated  In  ^er  to 
However,  the  task  represenffiS^^  Action  call, 

iheir  single  ask  load  values,  or  pass  die  se,uencedSs^S„^^nfv^ 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

3.3.1  Architecture 

calculatoer^a^?h^sertfnt^«C^^^I^ditorIclas^|t0rth*1C  adjus,or'  *' load 

dynamically  changes  ihe  classificadons  m a^^a  S,*'  lhe 

simulation.  The  load  calculator  generates  the  e c?ang®s  ,n  Ac  context  of  the 

.he  user  in.erface  drsplays  the  lofd  val„S.tKdtS  and 

3.3.2  Process  Capabilities 

one  to  n number  of  tasks^K^  simulation,  and  can  process  from 

Realistically,  the  TLM  would  SyTr^  *sk  P®*™**. 

Sr ‘singi anyi "°re *«*" 

3.3.3  Performance 

validation  will  ^urhf th^^xj ph^tfc ilveloprlSr56  “ haS"  * ^ teSteA  Testing  md 

3.3.4  Interfaces 

select  either  single-  or  multi-task  loading  vfii.Ac  ^f!i?^J^C  S on  the  screen.  designer  can 
values  of  the  tasks  and  task ^combinations' meLmbeni-  ^ fading 
classifying  the  task  (s),  the  name(s)  of  the  rasklsT  a n^rh!^8  W,th  the  taxonomic  subset 
ask  performance  was  simulated.  The  designer  ci  scroll  .he^is^SfallyS"  "* 
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vertically  to  display  the  complete  timeline  with  die  complex  task  classification  and  loading 
profiles. 

3.3.5  Error  recovery  capabilities 

As  of  yet  the  TLM  does  not  incorporate  models  of  human  emtr.  and  does  not  have  enor 

recove'ry  mechanisms  for  simulation  crashes  or  interruptions. 

3.3.6  Reliability 

The  reliability  of  the  TLMSC1  is  not  known  because  it  hasn't  been  tested  or  validated. 

3.3.7  Maintainability 

Cunendy  the  TLMSCI  can  be  “Sts  lSK SKS*.' thT* 

SSlSfn^e“ 

the  code  and  programming. 

3.3.8  Flexibility  and  Expansion 

The  TLM  is  currently  not  very  flexible Tro m Li^  machines. 

Lisp  and  runs  only  on  Lisp  machines,  a g P between  different  computers  is 

3.3.9  Transportability 

The  TLMSCI  is  not  a stand-alone  tool  ^s’JSrtabieS  the  MIDAS 

modeling  and  design  environment  As  , ^rks^tion  to  point  that  it  can  be 

applications,  but  when  this  will  occur  is  unknown. 

3.3.10  Quality 

•me  quality  of  the  TLMSCI  is  cuiremly  unknown,  and  will  remain  unknown  until  it  has 
been  tested  and  validated. 

3.3.11  Adaptation  to  Various  Operational  Sites 
me  TLMSCI 

to  specific  sites  will  have  to  be  done  by .the : use  . f<J  into  ^ TLM.  Once  the 

tasks  and  simulation  state  variables m n ^ J e(  Designers  would  be  required  to 

needed. 

3.3.12  Phased  Implementation 

The  TLM  results  from phase^he  T^Mwa^crea^^ 

was  just  finished  as  of  June  lyyu.  u g V j^e  next  phase  is  to  begin 

^to&^flwMl  continue  for  12  to  18  mondis.  During  diis  next 


Page  C-21 


tasks  and  adjust  task'  d^si^Sons3^^  wWdfin*6  expanf*l to  automatically  classify 
designer.  ’ tx>tn  of  which  are  currently  done  manually  by  the 

3.3.13  General  Flow  of  Data 

X !?££  r?  °to  M?LASmodd"  *»  Scheduler 

Model  (goal  decomposing  model).  P S tatl0n  nw^ods  in  the  Symbolic  Operator 

SwESESl  IffltSSSSX*  “ - - valuer 

^ssssass  -»■%«-  *.  m. 

simulated.  s necessary  to  sequence  the  tasks  that  need  to  be 

3.3.14  General  Flow  of  Execution  Control 

Operator  ModeLlS^  in  *e  Symbolic 

simulated,  and  in  doing  so  decides  when  each  task  nr  rrmr*  *asH  sefluences  t^at  will  be 

this  time,  the  TLM  is  called  to  evaluate the S^T**nC!TOt,t“ks  t*00™  active.  At 
Operator  Model  also  calls  the  TLM  mior  m rhp  «;  °f  ?!• actJ.ve  ,tasks-  The  Symbolic 
how  to  sequence  the  tasks.  The  Symbolic  OneramHVf^VT  03d  ValuCS  t0  help  decide 
help  decide  how  to  sequence  the  tasks  n ,dS°uCan  caU  **  Sch«iuler  to 

task  loading  values  the  Sch^u ler  Squire s ca?5  ^ 1LM  for  ^ 

scheduling  tasks.  The  potential  tMk  Iei^ent  a balancing  strategy  for 

Model  to  help  decide h«  fcSs"  by  'h'  Symbo“c 


3.3.15  Networking  Requirements 


medium! and  A.e  TLM  use  the  CHAOS 

Operator  Model  on  the  Symbolics  file  sernr ^ and^h^T!?3.16  bet'J^En  the  Symbolic 
Symbolics  machines.  ’ nd  tde  Scheduler  and  TLM  on  two  other 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 

begin  with,  tlfe  use/JmemSr^ of  a design  team)  f^u[!!'hin,fthe  MIDAS  environment  To 
would  enable  a pilot  to  fly  a specific  hme  of  mLsi™  S a ternate  cockP«  designs  that 
using  MIDAS  CAD  tools  Next  the  u£  S ??', USPr  *en  renders  *e  design 
describe  and  decompose  the  tasks  for  the  simulation  th£Sym.bo/lc  9Perator  Model  to 
subgoals  that  are  then  mapped  to  various  fliX  SitJ?C  ™s^lon  1S.  decomposed  down  to 
into  subactivities  that  are  mapped  to  their  reslwv  ^es’  w^?c^  ^ 1x1  tum*  decomposed 

rol°SS  " eq“ipmem  allo'«  ^ : SeSdes^TKwure 

f ""r  b“‘ « ■» 

lask  combinations  are  evaluated  usins  the  TLM  to  ....  .fa  °\r*  s|mulatton,  the  tasks  and 
and  Motor  (VACM)  loads  that  wo^d  be  i^ed  „Pn?nno,^^,SUal  •Audi,°1*  OwiWve 

0m'Uh'a>  deSig"S'  ,f  ,bc  <*«*>«'  v^es  for  a givPen  sqEcSSS, 
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^s^sz-sssssr' 

The  TLM  then  evaluates  the  loads  of  the ' ^Sct^S^aSStory, 

^S^S^dve  tasks  along  with  their  task  classifications. 

This  process  is  ***4 

KSSs^  whete  (he  .ask  combinations  required  unreasonable 
pilot  control  inputs. 

4.0  REQUIREMENTS 

~ws^^3SSSSssar*- 

2)  to  develop  a method  of  resource  description  that  is  general  enough  to  describe  any 
helicopter  aviation  task. 

3) ?eS°&1ea“ 

rvAPM^i  and  tasks. 


There  are ,o« co»«  TW:  1g£%%£3Z£2££i 'he 
Calculator,  and  the  User  I"terf^r  . • al  implementation  of  the  code  involves  an 

structure  of  the  software,  however,  t P y containing  the  data  object  types  and  the 
overlapping  file  stmcture.  mre  ax  g structures>  Methods,  and  functions)  Aat  arc 
operations  performed  on  them  (eg-. flavj 1 • separated  in  the  files  according  to 

BBS  SSre^  ba^  sep^aS  ac’Lding  •>  •**  operate*  .bar  are  performed  on 
the  data  objects. 

4 1 requirements  approach  and  tradeoffs 
KB  stares  can  be 


Page  C-23 


^?avi0R  “-  •»  “ «* 

HrHiS 

accessed  and  in  teerated  into  the  Tl  M mau*  ^.l,  • 111  • MIDAS  workstation  can  be 

contexts  in  tfte SSL"  'he  UM>  mala”6  *'  ”“««»“  » changing 

4.2  HARDWARE  ENVIRONMENT 

nMliSft^  H600  »ilh  3 MWoris 

and  a 17"  landscape,  monochrome  mom  tor  wfth  OclTfitoMd'  n 52  ™870  ?rfT'  Caid' 
resolution.  A minimum  of  8 megabytes  of  memory^  twmiimended^o  ran^the^M. 

4.3  SOFTWARE  ENVIRONMENT 

Smmon  {^w?U,*£^S ^^"0^  °“OT  72  Wto 

Dynamic  Windows  and  Presentation  Substrate  Systems.  programming  80(1  ^ Symbolics 

Smm^Lispf  I^^u^^^E^^inen^enTr  ” wi^do^n^M^irOTn^u 
networking  software,  and  Namesoace  svsrem  environment,  CHAOSnet 

T Pi'er’  IP-TCP  comScSn  Swale  4“  * 

FORTRAN Comp'S'  A"  thC  SOftWare  men“0“'d  is  used  **  *'  TLM  except  fold/ 

SSj^heiXinSg  fhe  n&  d“ing  developm™ «*>■«.  »u.  is  ntn  in  compiled 

4.4  EXTERNAL  INTERFACE  REQUIREMENTS 

(2)  a MIDAS  component  that  interfaces  with  S^AS  ‘hC  dCSign  tCam’ OT 

4.4.1  The  Designer's  Interface 

SncepSlfesig^  d“ig"erS  ,0  imeract  »“!■  >he  H.M  when  evalnating 

4.4.1.1  Purpose  of  the  Interface 

The  purpose  of  the  interface  is  to  facilitate  the  use  of  rhe  TtMtniuij'  , 

S^SSSKSSS ® by  pSSe  ? * “k  fiu®^  “d 

with  the  information  that  is  displayed.  * P d g the  user  Wlth  the  means  to  interact 

4.4.1.2  Information  Content  of  Displays 
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The  interface  displays  detail*  Mmta  M 

an  operator.  The  names  of  the  eont^phial  d loadings,  task  classifications,  and 

names,  psychological  dimensions  “1^  *ttnto  • environment  and  crew  members  are 

simulation  timeline  are  displayed.  Th  yp  J*1  displayed  for  each  crew  member, 

displayed  at  the  top  of  the  interface.  t^e.^n^SstSSSbel  for  each  load,  and 

SfS«s"ciSTe^t  hi  aiong  with  amibute  labels.  The  simulauon 

timeline  is  displayed  at  the  bottom  of  the  interface. 

4.4.1.3  Interacting  with  the  Interface 

The  user  interacts  with  the  display  with  f ^i^u'saSow fe  useno  Sldie  last 
items  and  whether  the  task 

load  data  for  each  crew  member,  for  suJBle  mouse  to  point  at  a 

load  data  should  be  re-calculated  or  re-  «P  y . i oa(j  and  task  classification 

scroll  bar  in  the  left  margin  to  scroll  margin  to 

4.4. 1.4  Interface  Constraints 

The  current  interface  is  ^orrfc^  the  application  of  this 

names  of  the  system  domain  the  The  user 

interface.  The  user  cannot  change  the  ta  displayed.  To  effect  any  changes, 

also  cannot  change  the  amount  of  • . underlying  code  and  then  recompile  it. 

SSSsSSSESHsifesssr 

environments, 

4 4 2 The  Interface  to  MIDAS  Components 

Thera  am  two  MIDAS  interfacing  systems  and  components  that  use  the  TLM:  the  Scheduler 
and  the  Symbolic  Operator  Model. 

4 4 2 1 Purpose  of  the  Interface 

components  that  require  them. 

4.4. 2. 2 Information  Content 

returns  the  load  values  associated  with  the  tasks. 

the  Symbolic  Operator  Model,  or  displays  them  on  the  TLM  interface. 
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4.4.2.3  Information  Flow 

myS  foT  10  flow  be,we'"  "*  Symbolic  Operator  Model,  the  Scheduler 


11  Mntel  B 7LM 10  Svmholir  Operator  Model-  used  when  the 

Symbolic  Operator  Model  needs  task  load  values  to  decide  task  sennm Th~  *0ou 

S'8  “ <*>“  >0  to  TIM,  which  hrt  (SiXuS  toeS 


2)  Qpgraior  Model  to  Scheduler  to  TLM  to  Scheduler  to  Symbolic  ryrnrnr 

Msm:  used  when  the  Symbolic  Operator  IZde 

ki  sequei?.ces-  J.he  d312  Passed  by  the  Symbolic  Operator  Moddtothe  * 

Scheduler  are  lists  of  instances  of  task  objects  and  constraints.  The  Scheduler 

eSal i*3*  ^ thtJL^  but  calIs  functions  that  classify  the  task  objects  and 

SS$£.l°‘ds-  ^ Scbeduler  ,h'n  p*8*8  Mck . Xhw  teSSSL 

31  to 

™ ta  ca"8  functo?tto  «to3»toSS5jS5-d 

4)  Symbolic  Operator  Model  tn  n 

used  to  display  the  task  load  values  for  the  task 

— 8 - P*88*  » to  “cf1 

4.4.2.4  Implementation  Constraints 

Currently,  the  TLM  cannot  obtain  infomiation  about  the  states  of  the  simulation 

Model.  Consequently,  in  Phase  IV,  all  classifications  are  done  mLually*  °pmtOT 

5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

SSSSc°toSLC0'^"'mS:  ,he  Task  Edi,0r  ■ to  Task  Adjustor,  die 

5.1.1  Design  Approach  and  Tradeoffs 

The  implementation  of  the  TLM  began  with  developing  the  TLM  Calculator  This 
Bushnell,  as  die  author  a,  .he  toe  was  no,  a programmer.  The  authTSuS  Eeturram 
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unplentoutionofto™^ 

code  was  developed. 

5.2  DETAILED  DESIGN 

^o™*Sto^ 

pr  1 ^ Three  files  contains  the  code  that  implements  the  Task  Editor 

aass.”' — ss&i.  j *.  >**—*■' "•» 

Scheduler  . _ M 

gg  £SaK5£%*  - • *— 

This  section  on  tie, ailed  design  describes  to  code,  logic  fio»  arid  connpUation  ends  . .he 
seven  functional  groups. 

5.2.1  FGl:  The  .ask  edi.or  and  data-inpu.  functional  group 

FILE:  SQUID:>stav>midas-tlm>task-names.lisp 

DESCRIPTION:  This  file  contiuns  .he  functions  and  hash  tiibles  contiuning  to  names  and 
indices  of  all  the  tasks  used  in  the  simulation 

VARIABLES  DEFINED:  *CPG'T ASK-NAME-TABLE*  ’PILOT-T ASK-NAME ■ 

Parameters  defined: cKMJC^Nj^ P£!^iroR?NAM^CPa- 

hash  for  sh0" rames  *'  pao<'s 

♦ oclf  ^ 

INITIAL-VALUE:  nil 

SSSSSS*  hash  table  for  .he  shor,  nanKS  of  to  cpg's 

toclrc 

INITIAL-VALUE:  nil 

to  shod  names  for  to  long  msk  names 

for  the  pilot 

INITIAL- V ALUE : List  of  pilot  task  long-names 
NAME:  CPG-SHORT-NAMES-TABLE 
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fS  fcCS^PN:  ”akes  the  hash  Bbte  containing  the  shot,  names  for  the  long  ask  names 
:size  200 

INITIAL- VALUE:  List  of  copilot  task  long  names 
NAME:  PILOT-SHORT-NAME 

^IT^^tSm^table*  °f  3 l0ng  tesk  narae  from  PHot-task-name-table 
NAME:  CPG-SHORT-NAME 

INp' [Tref*cj)g-  tasl^n  ame  Stable*  ^ of  a lonS task  nan*  from  cpg-task-name-table 

NAME:  PILOT-LONG-NAMES 

variable  pilot-ion gmames  ^ °f  long  ,iames  for  Pilot’s  tasks  and  assigns  it  to  the 

INITIAL-VALUE:  nil 


NAME:  CPG-LONG-NAMES 

cpg-long-names  CreatCS  the  list  of  long  names  for  cpg  tasks  and  assigns  it  to  the  variable 
INITIAL-VALUE:  nil 

FILE:  SQUID:>stav>midas-tlm>tIm-task-Ioad-data.Iisp 

DATA-TABLE*  K LUAD-DATA-TABLE*  *PILOT-TASK-LOAD- 

'^SK^Sd^E^^^O^Tj^^E^GET^ cpg  T^^NAIV^^pS^^^  get-pilot- 

PILOTTASK-NAMES-FROM  TART  P rPT '^K;^^fES*FROM-TABLE  GET- 
AC.VE-TASKS  fpS-K 

de^ttS!S'I^sk'L0^:^ta'table* 

their  load  indices  for  the  pflot  lnitializes  the  hash  tabIe  containing  the  long  task  names  and 
INITIAL- VALUE:  nil 

NAME:  *CPG-TAS  K-LOAD-DATA-TABLE * 

lhe  hash  ,able  “"•nining  *e  long  ask  names  and 

INITIAL- VALUE:  nil 

NAh^:  PILOT-TASK-LOAD-DATA-TABLE 

indices  ford^pdo^3165  **  h3Sh  131)16  con,aining  the  Iong  task  names  and  their  load 


•irdtid-contents:  list  of  pilot  task  names  and  indices 

NAME:  CPG-TASK-LO/^-DAT^  names  and  their  load 

DESCRIPTION:  creates  the  hash  table  containing  in  & 

indices  for  the  copilot 

•SaLeontents:  list  of  copilot  task  names  and  indices 

NAME:  GET-P^OT-ACnVE-TASKS  ^ from  the  *pilot-task- 

DESCRIPTION:  gets  a list  of  the  task-loaa  inaices 

load-data-table* 

load-data-table* 

SWS^'^^Spw^tomb^task-loads 

• *pilot-task-load-data-table* 

FUNCTIONS  CALLED:  maphash 

NAME:  GET-CPG-TASK-NAMES-FROM-ML£  _ * task.load.data-uble* 

INSCRIPTION:  gets  the  keywords  from  the  hasn  taoie 
FUNCTIONS  CaLlED:  maphash 

- hite  from  ft.  hash  .able  .cpg-.ash-load- 

FILE:  SQUID :>stav>midas-tlm>tlm-editor.lisp 

DESCRIPTION:  ™^le“n.K‘ins IS^JlXhe T^'Sose'of  extensive  changes. 
SS5SSS  2 6:  USER,  GUIDE 

Foturc  changes  and  versions  will  modtfy  this  file  to  search  for  input  dam  values  in  the  data 
pools  distributed  throughout  MIDAS. 

5 2.2  FG2:  The  task-load  calculator  functional  group 
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FILE:  SQUID:>Stav>midas-tlm>tlm-calculator.lisp 

AUTHOR:  Hank  Bushnell 

calculate  the  task  load  values  **  functions’  constants>  ^ variables  required  to 

f°r 

Vl1u^o?^^?^OTOR' 

LOAD  WITHIN-SUM-OVER-TASKS  J^SUAL-AUDITORY- 

SUM-OVER-ONE-TAS  K H^OVERTASKS- 1 WITHIN- 

wrmiN-Au^TORY -lo^J^ 

SYM-ARRAY  MA^SWaSKS  PWNT-2D-ARRAY  PRINT- 
NAME:  ASET-SYM 

f 2 Sy,mKak  « is  set 

CALLING  FUNCTIONS:  MAKE-SYM-ARRAY 

NAME:  MAKE-SYM-ARRAY 
IWU^^RF^PRCT^i  initializes  a symmetric  array. 

otouts*“m  array  1nitial-contents 

FUNCTIONS  CALLED:  ASET-SYM 
NAME:  PRINT-SYM-ARRAY 

NAME:  PRINT-2D- ARRAY 

K™ £"2*  °ut  a ^-dimensional  array. 

INPUTS:  ARRAY  &OPTIONAL  (STREAM  T) 

NAME:  *VISUAL-CM* 

n?atrix  for  the  Visual  dimension 
ALUE.  A matrix  of  conflict  and  demand  values’ 

NAME:  * AUDITOR  Y-CM* 
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nFSCRIPTION:  The  conflict  matrix  for  the  Auditory  dimension. 

INITIAL-VALUE:  A matrix  of  conflict  and  demand  values 

SESM^S^Shct  matrix  for  the  Cognitive  dimension. 

A matrix  of  conflict  and  demand  values 

DESQUFn^^econflict  matrix  for  the  Motor  dimension. 

SSalTv  ALUE:  A matrix  of  conflict  and  demand  values 

DES^fl^Ol^  f°r  the  Visual- Auditory  dimension. 

INITIAL-VALUE:  A matrix  of  conflict  and  demand  values 

NAME:  *VISUAL-CCX3NTnVE-CM  Visual-Cognitive  dimension. 

nr^PRIPTION*  The  conflict  matrix  for  the  Visual 

INITIAL-VALUE:  A matrix  of  conflict  and  demand  values 

NAME:  *AUDrTOTY-C<^NITI^-CM  t0ry-C0gnitiVe  dimension. 

for  the  Cogninve-Motor  dimension. 
INITIAL-VALUE:  A matrix  of  conflict  and  demand  values 

NAME:  TASK-ELEMENTS  (structure) 

DESCRIPTION:  Structure  descnbing  the  elements  of  a 
INITIAL-VALUE:  '(visual  auditory  cognitive  motor) 

NAMF-  WITHIN- VISUAL-LOAD  _ , 

DESCRIPTION:  Calculate  the  visual  load  of  a set  of  tasks. 

INPUTS:  ELEMENTS -OF-TASKS  VISUAL  WITHIN-SUM-OVER-ONE- 

■l^SS as®" 

MAMF-  WTTHIN-AUDrrORY-LOAD 

DESCRIPTION:  Calculate  the  auditory  load  of  a set  of  tasks. 

INPUTS:  ELEMENTS-OF-TASKS  AUDITORY  WTTHIN-SUM-OVER- 

SSSHssr” 

NAMF-  WITHIN-COGNITIVE-LOAD 

DESCRIPTION:  Calculate  the  cognitive  load  of  a set  of  tasks. 

INPUTS:  ELEMEOTS-OF-TASKS  WITHIN-SUM-OVER- 

NAME’  WTTHIN-MOTOR-LOAD 

DESCRIPTION:  Calculate  the  motor  load  of  a set  of  tasks. 


Page  C-31 


INPUTS:  ELEMENTS-OF-TASKS 
TASKJT^^ 

CALLING  FUNCTIONS:  MOTOR-LOAD  cSSS™’2 
NAME:  WITHIN-SUM-OVER-ONE-TASK 

Sum  lhC  contribu,io'’s  ,0  a '“0  ton.  one  tasks:  alt.k,i]*«[tj,i])  + 

TAsiSSi  jLkNFLICI"matru<  task-elements-accessor  ELEMENTS-OF- 

NAME:  WITHIN-SUM-OVER-TASKS-1 

(aftSr^Jj,?])1'  SUm  thC  C°lumn'Wise  contnbutions  to  a load  from  all  tasks 

TASK  ELEMENTS-ACCESSOR  ELEMENTS-OF- 

NAME:  WITHIN-SUM-OVER-TASKS-2 
NAME:  VISUAL-AUDITORY-LOAD 

load  for  a « »f  “fa- 

NAME:  VISUAL-COGNTITVE-LOAD 
tas™^^^ 

CALLING  FUNCTIONS:  COGNITIVE-LOAD  GENERATE-LOAD 
NAME:  AUDITOR Y-COGNOTVE-LO AD 

INIHLU)!rpLI^E^sPoFt-^TASKS0ry<0gn'tiVe  '°ad  f°r  3 “ of  ,asks 

CTON1^  setween-sum-over- 

CALLING  FUNCTIONS:  AUDITORY-LOAD  COGNITIVE-LOAD  GENERATE-LOAD 
NAME:  COGNITIVE-MOTOR-LOAD 

WHTOES^^?fJ®‘,e-,,,0Wr  '°ad  for  a “* of 
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CALLING  FUNCTIONS:  COGNITIVE-LOAD  MOTOR-LOAD  GENERATE-LOAD 
NAME:  BETWEEN-SUM-OVER-TASKS-1 

DESCRIPTION:  Sum  the  row-wise  contributions  to  a load  irom  an  me 
^putskcot^icr-MATRix  row-task-elements-accessor  col-task- 

AUDITORY-COG  NITTVE-LO  AD  COGNITTVE-MOTOR-LOAD 
NAME:  BETWEEN-SUM-OVER-TASKS-2 

DESCRIPTION:  Sum  the  column-wise  contributions  to  a load  from  all  tne 

^^l^S^  CC^rtScT-MATRIX  ROW-TASK-ELEMENTS-ACCESSOR  ELEMENTS- 

^ALLING  FUNCTIONS:  VISUAL- AUDITORY-LOAD  xaSU^-COGNTITVE-LOAD 
AUDITORY-COGNITIVE-LOAD  COGNITIVE-MOTOR-LOAD 

DESCRIPTION:  Calculate  the  average  of  the  sums  of  the  three  possible  sources  of  visual 
loads. 

FWOTONS^^D^WmilN-VISUAL-LOAD  VISUAl^AUDITORY-LOAD 
VISUAL-COGNITIVE-LOAD 
CALLING  FUNCTIONS:  GENERATE-LOAD 

DESCRIPTION:  Calculate  the  average  of  the  sums  of  the  three  possible  sources  ofloads. 

FUNC^ONS^ALLED^WnWN-AUDITORY-LOAD  VISUAL-AUDITORY-LOAD 
AUDITOR  Y-COGNTITVE-LO  AD 
CALLING  FUNCTIONS:  GENERATE-LOAD 

NAME: COGNITIVE-LOAD  ...  , 

DESCRIPTION:  Calculate  the  average  of  thesums  of  the  four  possible  sources  ot 

cognitive  loads. 

IWJCnONS^ALEED^W^nN^CCXjNrnVE-LOAD  AUDrTORY -COGNITIVE- 
LOAD  VISU^-COGNITIVE-LOAD  COGNITIVE-MOTOR-LOAD 
CALLING  FUNCTIONS:  GENERATE-LOAD 

DESCRIPTION:  Calculate  the  average  of  the  sums  of  the  two  possible  sources  of  motor 
loads. 

^CTb^cS^f^N-MaiOR-LOADCOGNrnVE-MOTOR-LOAD 

CALLING  FUNCTIONS:  GENERATE-LOAD 

NAME:  GENERATE-LOAD  , . 

DESCRIPTION:  takes  one  or  more  task  objects  as  its  argument,  and  prints  out  the 

respective  individual  and  multiple  task  loads 

OUTPUTS'  '(WITH IN- AUDITORY-LOAD  WITHIN-COGN 1T1VE-LOAD  V/n'HIN- 
MOTOREOAD  VISU  AL- AUDITORY -LOAD  AUDITORY  -COGNITIVE-LOAD 
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COGNTITVE-MOTOR-LOAD  VISUAL-LOAD 
AUDITORY-LOAD  COGNITIVE-LOAD  MOTOR-LOAD) 

<-'ALLED:  (READ-INSTANCE-VARIABLE  INDEX- V TASK  INDEX- V) 
(READ-INSTANCE-VARIABLE  INDEX-A  TASK  INDEX-A)  (READ-INSTANCE 

Ya  : Yi'SXASK  (READ-INSTANCE- VARIABLE  INDEX-M 

TASK  INDEX-M)  WITHIN- VISUAL-LOAD  SCALE-LOAD- VALUES  WITHIN- 
AUDI  TORY -LOAD  WITHIN-COGNITIVE-LOAD  WITHIN-MOTOR-LOAD  VISUAL- 
A^rTORY-IXMD  AUDITORY-OOGNmVE-LOAD  VISUM^f^-I^ 

LOAD^OTOT^LO^^^  VISUAL-LOAD  AUDITORY-LOAD  COGNITIVE- 

CALLING  FUNCTIONS:  SHOW-TASK-LOAD  DISPLAY-SINGLE-TASK-LOADS 
DISPLAY-COMBINED-TASK-LOADS  MULTIPLE-TASK-LOAD  SCHEDULER- 
I -O  AD-  LIST 


NAME:  SCALE-LOAD- VALUES 
DESCRIPTION:  scales  the  loading  values 
INPUTS:  LOAD- VALUE 
OUTPUTS:  SCALED- VALUE 
CALLING  FUNCTIONS:  GENERATE-LOAD 

5.2.3  FG3:  The  display  windows  and  menus  functional  group 


FILE:  SQUID:>stav>midas-tlm>task-load-interface-frame.lisp 

DESCRIPTION:  This  file  contains  the  flavors  and  methods  required  to  create  the  windows 
and  panes  that  are  used  to  display  the  task  load  data 
VARIABLES  DEFINED:  *HW1* 

FLAVORS  DEFINED:  TASK-LOAD- WINDOW-FRAME  TIME-LINE-LABEL-PANE 
TIME-LINE-PANE  TASK-COMBINATION-LOAD  PANE  SINGLE-TASK-LOAD- 
PANE 

TASK-LOAD-PANE-PARENT  TASK-ELEMENT-LABEL-PANE  ELEMENT-TITLE- 

nJ^PANE  CREW-SELECTION-MENU-PANE  TLM-TITLEPANE 
Mb 1 HODS  DEFINED: 

(DISPLAY-MOTOR-ELEMENT- VALUES  TASK-LOAD-PANE-PARENT) 

S!cd(jw'^9Jor'element'labelstask‘element-label-pane) 

(D  SPLAY-COGNITIVE-ELEMENT-VALUES  TASK-LOAD-PANE-PARENT) 
(DISPLAY-CCXINTTiVE-ELFMFNT-^BELS  TASK-ELEMENT-LABEL-PANE) 
(DISPLAY- AUDITORY-ELEMENT-VALUES  TASK-LOAD-PANE-PARENT) 
(DISPLAY-AUDITORY-ELEMENT-LABELS  TASK-ELEMENT-LABEL-PANE) 
(DISPLAY- VISUAL-ELEMENT- VALUES  TASK-LOAD-S-PAmJ^  ^ 
S!^,LAY  VISUAL-ELEMENT-LABELS  TASK-ELEMENT-LABEL-PANE) 
(DISPLAY-VACM-LOAD-VALUES  TASK-LOAD-PANE-PARENT) 

(DISPLAY- VACM-LOAD-LABELS  TASK-ELEMENT-LABEL-PANE) 
(DRAW-TIME-LINE  TIME-LINE-PANE)  ’ 

(DISPLAY-CREW-LABEL  ELEMENT-TITLE-PANE) 
(DISPLAY-TASKCOMBO-LABELS  TASK -TITLE-PANE) 

(DISPLAY-TASK-LABELS  TASK-TITLE  PANE) 

(PRINT-THE-INITIAL-DISPLAY  TASK-LOAD-WINDOW-FRAME) 
(PRINT-TO-TASKCOMBO-ELEMENT-AND-LOAD-PANE  TASK-LOAD- WINDOW- 

FRAME) 

(PRINT-TO-TASK-ELEMENT-AND-LOAD  PANE  TASK-LOAD- WINDOW-FRAME) 

( Y-SCROLL-TO  TASK-ELEMENT-LABEL  PANE  Al  'IER) 

(X-SCROLL-TO  SINGLE-TASK-LOAD-PANE  ALTER) 
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!aMoc!atedJomb1ned-load-pane  * 

I ASSOCIATCD-LO AD-PANE  TASK-ELEMENT-LABEL-PANE) 
qNTT  TASK-LOAD-WINDOW-FRAME  AFTER) 

SeSotSn:  task-load-window-frame  object  initialized  to  nil 
INITIAL- VALUE:  NIL 

SSSSSKSSSE  top  column  pane  confining  the  ode  "MIDAS  Task  Uad 

otSStHJ^ORS:  DYNAMIC-WINDOW-PANE 

NAME:  CREW-SELECTION-MEW-PANE  ^ ^ sensitive  crew 

DESCRIPTION:  Defines  the  second  column  pane  containing 

Component  flavors:  top-label-mixin  centered-label-mixin 

COMMAND-MENU-PANE 

iCOLUMNS  3 

:ITEM-LIST  *CREW-MENU* 

row  pane  of  the  third  column  panes  containing  the  task 

COMBINATlON-LOAD- 

*M^mOD^'DlSPLAYrrASKCOMBO-L.ABELS  DISPLAY-TASK-LABELS 

umw  pane  of  the  thud  column  panes  containing  the 

toSoS^Svors:  dynamic-window-pane 
METHODS:  DISPLAY-CREW-LABEL 

DESCRIP^Ol^^hneshie^Uuow"  pantfof  the  fourth  column  panes  containing  the  dm 
SI«  SSSSSSSSKESSS  TASK-COMBINATION- 

associated-load-pane 

NAME:  TASK-LOAD-PAI^PARENT  pANE  d TASK-COMBINATION- 

DESCRIPTION:  Parent  of  SINGLE-TASK-LUAU  r/uNc  anu  ^ 

W^ONEOT  FLAVORS:  DYNAMIC-WINDOW-PANE 
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“/S.PANErFLAV0RS:  TASK-combINATION.LOAD-PANE  SINGLE-TASK- 

INSTANCE  VARIABU:S:  TICK-SCALE  rimtial-value  5 
TICK-SPACING  .initial-value  5 
ROW-VSP  .initial-value  5 
START-TIME  :initial-value  9999999000 
DEFAULT-START™  :initial-value  0 
END-TIME  :initial-value  -99999999 
DEF AULT-END-UME  : initial-value  200 

NAME:  SINGLE-TASK-LOAD-PANE 

oomponi^  for  sin*1'  asks 

NAME:  TASK-COMBINATION-LOAD-PANE 

S*  T^K^/Spa^ArSt  f°rCOn’bined  asks 

STANCE  VARIABLES:  TASK-TITLE-PANE  TIME-LINE-PANE 

NAME:  TIME-LINE-PANE 

SS t“skBti^eSpan^‘TASK‘L0ad'pane  task-oombination- 

“SOC'AWOOMBINED-TTILE- 

NAME:  TIME-LINE-LABEL-PANE 

disPla>’s  the  label  for  the  time-line-Dane 

COMPONENT  FLAVORS:  DYNAMIC- WINDOW-PANE 

NAME:  TASK-LOAD- WINDOW-FRAME 

COMPONEUT  FLAVORS:  BORDERED-CONSTRAINT-FRAME-WITH-SHARED-IO 

TASK^n^&PAT^^^EMEf^^rTLE-pANE  WLO?*SIN^?tac^?^^"PA^ 

SSsSr® 

smnsnr 

task-cbmbination-load-pane  cm  iA?FLm^Kh°^rA^PIL0T- 
PILOT.TIME.LINEPANECPG^.LMPAra  ° L°AD‘PANE 
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INCREMENT- VALUE-COLUMN-PILOT  :initial-value  4 
INCREMENT- VALUE-COLUMN-CPG  :initial-value  4 
INCREMENT-LABEUCOLUMN-PILOT  :initial-value  3 
ENCREMENT-LABEL-COLUMN-CPG  :initial-value  3 
STRING-XPOS-PELOT  :initial-value  4 
LINE-XPOS-PILOT  :initial-value  32 
STRING-XPOS-CPG  :initial-value  4 
LINE-XPOS-CPG  :initial-value  32 
TICK-HEIGHT  Initial-value  53 

^o^SpLS™:S^-display  PRINT-TO-TASKCOMBO-ELEMENT- 
AND-LO  AD-PANE  PRINT-TO-TASK-ELEMENT-AND-LO  AD-PANE  INIT 
CONFIGURATIONS  '((configl  config2  config3  config4)) 

NAME:  (FLA VOR:METHOD  INTT  TASK-LOAD- WINDOW-FRAME) 

DESCRIPTION:  assigns  the  names  of  the  instance  variable  panes  their  associated 

instances  of  the  panes  and  lets  instances  of  the  panes  of  each  instance  variable  know  about 
instances  of  panes  associated  with  it 
INPUTS:  &REST  IGNORE 

NAME:  (FLAVOR:METHOD  ASSOCIATED-LOAD-PANE  TASK-ELEMENT-LABEL- 

^DESCRIPTION:  gets  the  panes  associated  with  scrolling  task-element-label-pane  for  the 
appropriate  configuration 

NAME:  (FLAVOR:METHOD  ASSOCIATED-TTTLE-PANE  SINGLE-TASK-LOAD- 
PANE) 

DESCRIPTION:  gets  the  panes  associated  with  scrolling  single-task-load-pane  for  the 
appropriate  configuration 

NAME:  (FLAVOR:METHOD  ASSOCIATED-COMBINED-LOAD-PANE  TIME-LINE- 

PANE)  .... 

DESCRIPTION:  gets  the  combined  task  load  panes  associated  with  scrolling  time-line- 

pane  for  the  appropriate  configuration 

NAME:  (FLAVOR:METHOD  ASSOCIATED-COMB INED-TITLE-PANE  TIME-LINE- 

PANE)  ... 

DESCRIPTION:  gets  the  combined  task  title  panes  associated  with  scrolling  time-line-pane 

for  the  appropriate  configuration 

NAME:  (FLAVOR:METHOD  X-SCROLL-TO  TIME-LINE-PANE) 

DESCRIPTION:  scrolls  pilot-task-combination-load-pane  and  pilot-combined-tasK-tiue- 
pane  if  its  config3  or  cpg-task-combination-load-pane  and  cpg-combined-task-title-pane  it 
its  config4  when  time-line-pane  is  scrolled 
INPUTS:  &REST  ARGS 

NAME:  (FLAVOR:METHOD  X-SCROLL-TO  SINGLE-TASK-LOAD-PANE) 
DESCRIPTION:  scrolls  pilot-single-task-title-pane  if  its  configl  or  cpg-single-task-title- 
pane  if  its  config2  when  either  pilot-single-task-load-pane  or  cpg-single-task-load-pane  is 
scrolled 

INPUTS:  &REST  ARGS 

NAME:  (FLAVOR:METHOD  Y-SCROLL-TO  TASK-ELEMENT-LABEL-PANE) 
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DESCRIPTION,  scrolls  pilot-singlc-task-load-pane  if  itsconfigl  or  cpg-single-task-load- 
pane  if  its  config2  or  pdot-task-combination-load-pane  if  its  config3  or  cpg-task- 

C^uTs°&^STaARGS  COnfig4  When  ^sk-element-label-pane  is  scrolled 

'^•TO-TA“-aEMEOT'.AND.LOAD-PAIffi  TASK- 

■ SSC1  • ^ «* 

NAME;  (FLAVORrMETHOD  PRINT-TO-TASKCOMBO-ELEMENT- AND- LOAD- 
PANE  TASK-LOAD-WINDOW-FRAME) 

DESCRIPTION:  displays  task  load  data  for  task  combinations 
INPUTS:  TASK-OBJECTS  CURRENT-TIME  TASK-NAMES 

W^W^FRA^^™00  PRINT-THE-INITIAL'D^PLAY  TASK-LOAD- 

for^se^tbg^^display  **  'dnd  paneS  With°Ut  load  valucs  * used 

npS™0?^?1!  DISPLAY-TASK-LABELS  TASK-TITLE-PANE) 
DESCRIPnON.  displays  the  labels  for  each  task  in  the  task  title  pane 
INPUTS:  INCREMENT-COLUMN  TASK-LABEL 

DFSTR  ^SPLAY-TASKCOMBO-LABELS  TASK-TITLE-PANE) 

ixmr  cksPlays  tfie  labels  for  each  task  combintion  in  the  task  title  pane 

INPUTS:  INCREMENT-COLUMN  TASK-NAMES  P 

DF^RT^nM0?^™?0  DISPLAY-CREW-LABEL  ELEMENT-TTILE-PANE) 

SS^Sw^LABEL16  CrCW  abC  m the  taSk  titlC  panC 

n^T^YPRu:METH0D  DRAW-TIME-LINE  TIME-LINE-PANE) 

in  S th,s  £uenes  Pe  taskLindex  files  for  changes  in  the  file  upon  which  it  reads 

rnthe  time  and  draws  the  time  line  in  the  time  line  pane  and  labels  each  tick  with  the  current 

INPUTS:  STRING-XPOS  LINE-XPOS  CURRENT-TIME 

l^»R“D  displayvacmload-labels  task-element- 

pos'"on  and  fon"a,  for  -*  vacm  «*— ■-* 

««  DISPLAY- VACM-LOAD- VALUES  TASK-LOAD-PANE- 

SrSd^n,wtSc“ri0r  P°Sili0n  a”d  f0r“Ch  vacm  component  load 
INPUTS:  OBJECT  INCREMENT-COLUMN 

ENli^tS^VBa-M™D  DISPLAY-VISUAL-ELEMEm--LABELS  TASK- 

P“i,i0"  “d  fon“  foreach  Yisual  componem  “ 
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NAME  (FLAVOR:METHOD  DISPLAY- VISUAI^ELEMENT- VALUES  TASK-WAD- 

MSOTS*«n»ines  the  cursor  position  and  forma,  for  each  visual  component  load 

^s^ctTncreScolumn 

NAME:  (FLAVOR:METHOD  DISPLAY-AUDIT ORY-ELEMENT-LABELS  TASK- 

the  cursor  posinon  and  forma,  for  each  auditory  component 
load  label  in  the  task-element-label-pane 

NAME-  (FLAVORtMETHOD  DISPLAY-AUDITORY-ELEMENT-VALUES  task- 

LD°E^RP^NASnes  the  cursor  position  and  fotma,  for  each  auditory  component 

load  value  in  the  task-element-label-pane 
INPUTS:  OBJECT  INCREMENT-COLUMN 

NAME-  (FLAVORtMETHOD  DISPLA Y-COONinVE-ELEMENT-LABELS  TASK- 
SraSS  the  cursor  position  and  forma,  for  each  cognitive  component 
load  label  in  the  task-element-label-pane 

NAME:  (FLAVORtMETHOD  DISPLAY-COGNITIVE-ELEMENT-VALUES  TASK- 
M^MON^Sines  the  cursor  position  and  forma,  for  each  cogninve  component 

load  value  in  the  task-element-label-pane 
INPUTS:  OBJECT  INCREMENT-COLUMN 

NAME:  (FLAVOR:METHOD  DISPLAY-MOTOR-ELEMENT-LABELS  TASK- 

ELEMENT-LABEL-PANE)  fnrmat  for  each  motor  component  load 

DESCRIPTION:  determines  the  cursor  position  and  lorm 

label  in  the  task-element-label-pane 

NAME:  (FLAVORtMETHOD  DISPLAY-MOTOR -ELEMENT- VALUES  TASK-LOAD- 
^DE^RIRnON?  determines  the  cursor  posinon  and  forma,  for  each  motor  component  load 


FILE*  SQUID:>stav>midas-tlm>select-task-display.lisp 

VARIABLES  DEFINED:  *CREW-MENU  DISPLA  YSELECT-CPG-DISPLAY- 

PARAMETERS  DEF^D^ELECT-RESE^  SELECT-CPG- 

TASK-DISPLAY  cxadt  rpr,  PROCESS  START-PILOT-PROCESS 

^IH^L^^CT^T^^-CO^^^^nONS^  CPG-TASK-COMBINATTON^^ 
MDi1pLAY*-CPG-SIN^I^*^A^K^dSh^AY-CPG-SINGLE-TASKS  REDISPLAY- 
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DISPLAY-PILOT-SINGLE-TASKS  PHFrK-  pn  p r-riD 

NAME:  ♦CREW-MENU* 

the  menu  for  ±c  crew-selection-mcnu-pane 

•documentation NO-SELECT  NIL)  M KtW  SELECTION  MENU  .value  erase 
SELECT  $£?  VALUE  PIL0T  DOCUMENT ATION  Selects  the  Pilot’s  Tasks)  ( NO- 
Tasks))  (Copilot/Gunner  VALUE  COPILOT  DOCUMENTATION  Selects  the  Copilot’s 

NAME:  SELECT-RESET-TLM-DISPLAY 

n.M  pS°N:  Crea‘“ th'  P°P'UP  m“U  10  C)5ar  lhe  d,sP‘a>'s  ®1  restart  MIDAS- 
INITIAL- VALUE:  .MAKE-WINDOW  'MOMENTARY-MENU 
■LABEL  How  about  a nice  game  of  chess 
BORDERS  3 

’ITEM-LIST  ’("RESET-TLM") 

FUNCTIONS  CALLED:  RESET-TLM  ' 

NAME;  SELECT-PILOT-TASK-DISPLAY 

"*  P°P'“P  10 Sdea  thc  p"°*  «■  combined  ask 

INITIAL-VALUE:  :MAKE-WINDOW  'MOMENTARY-MENU 
'LABEL  Select  Display 
'BORDERS  3 

NAME:  SELECT-CPG-TASK-DISPLAY 

CreatM  P0P‘UP  mm  10  ,he  •*$  single  or  combined  ask  display 

INITIAL-VALUE:  : MAKE -WINDOW  'MOMENTARY-MENU 
'LABEL  Select  Display 
'BORDERS  3 

shlect-pilot-display-type-single 

LABEL  Select  Display 
'BORDERS  3 
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NAME:  SELECT-CPG-DISPLAY-TYPE-SINGLE  ...  , 

DESCRIPTION:  creates  the  pop-up  menu  to  select  the  cpg  single  task  display 
INITIAL-VALUE:  :MAKE-WINDOW  MOMENTARY-MENU 
LABEL  Select  Display 
'BORDERS  3 

'ITEM-LIST  ’("Calculate  Loads"  "Redisplay  Loads  ) 

FUNCTIONS  CALLED:  DISPLAY-CPG-SINGLE-TASKS  REDISPLAY-CPG- 
SINGLE-TASKS 

NAME:  SELECT-PILOT-DISPLAY-TYPE-COMBINED 

DESCRIPTION-  creates  the  pop-up  menu  to  select  the  pilot  combined  task  display 
INITIAL-VALUE:  :MAKE-WINDOW  MOMENTARY-MENU 
'LABEL  Select  Display 

'BORDERS  3 , , . „ 

'ITEM-LIST  '("Calculate  Loads"  "Redisplay  Loads  ) 

FUNCTIONS  CALLED:  check-file-for-active-pilot-tasks  redisplay-pilot-task- 
combinations 

NAME:  SELECT-CPG-DISPLAY-TYPE-COMBINED 

DESCRIPTION-  creates  the  pop-up  menu  to  select  the  cpg  combined  task  display 
DOTTALVALUE:  :MAKE-W1ND0W  MOMENTARY-MENU 
'LABEL  Select  Display 
’BORDERS  3 

TTEM-LIST  '("Calculate  Loads"  "Redisplay  Loads  ) . „ 

FUNCTIONS  CALLED:  CHECK-FILE-FOR-ACTIVE-CPG-TASKS  REDISPLAY- 
CPG-TASK-COMBINATIONS 

NAME:  RESET-TLM-DISPLAY  . , ,lc 

DESCRIPTION:  sends  a message  to  the  defparameter  select-reset-tlm-di splay  that  cal  s 
the  pop-up  menu  that  resets  the  "‘tlm-process*  to  the  initial  state 
CALLING  FUNCTIONS:  MIDAS -TLM-DISPLAY 

NAME:  PILOT-TASK-DISPLAY  . ..  . 

DESCRIPTION:  sends  a message  to  the  defparameter  select-pilot-task-display  calls  the 
pop-up  menu  that  selects  either  the  pilot's  single  or  combined  task  display 
CALLING  FUNCTIONS:  MIDAS -TLM-DISPLAY 

NAME:  PILOT-DISPLAY-TYPE-SINGLE  . . t.  t 

DESCRIPTION:  sends  a message  to  the  defparameter  select-pilot-display-type-single  that 
calls  the  pop-up  menu  that  selects  pilot  single  task  display 

NAME:  PILOT-DISPLAY-TYPE-COMBINED  . . 

DESCRIPTION:  sends  a message  to  the  defparameter  select-pilot-display-type-combined 
that  calls  the  pop-up  menu  that  selects  pilot  combined  task  display 

NAME*  CPG  TASK-DISPLAY 

DESCRIPTION:  sends  a message  to  the  defparameter  select-cpg-task-display  that  calls  the 
oop-up  menu  that  selects  either  the  cpg's  single  or  combined  task  display 
CALLING  FUNCTIONS:  MIDAS-TLM-DISPLAY 

NAME:  CPG-DISPLAY-TYPE-SINGLE 

DESCRIPTION:  sends  a message  to  the  defparameter  select-cpg-display-type-single  that 
calls  the  pop-up  menu  that  selects  cpg  single  task  display 
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NAME:  CPG-DISPLA Y -TYPE-COMBINED 

DESCRIPTION:  sends  a message  to  the  defparameter  select -cpg-display-typc -combined 
that  calls  the  pop-up  menu  that  selects  cpg  combined  task  display 

NAME:  CHECK-FILE-FOR-ACnVE-PILOT-TASKS 

DESCRIPTION:  initializes  the  active  pilot  task  file  and  starts  the  process  "Combined  Pilot 
Task  Loads" 

FUNCTIONS  CALLED:  SCL:PROCESS-RUN-FUNCTION 
NAME:  CHECK-FILE-FOR-ACTTVE-CPG-TASKS 

DESCRIPTION:  initialize  active  cpg  task  file  and  starts  the  process  "Combined  CPG  Task 
Loads" 

FUNCTIONS  CALLED:  SCL:PROCESS-RUN-FUNCTION 


NAME:  DISPLAY-PILOT-SINGLE-TASKS 

DESCRIPTION:  calls  all  the  functions  necessary  to  display  the  pilot's  single  task  load 
values  and  sets  the  configuration  of  *hwl * to  'configl 
FUNCTIONS  CALLED:  DISPLAY-PILOT-SINGLE-TASK-LOADS 

NAME:  REDISPLAY-PILOT-SINGLE-TASKS 

DESCRIPTION:  redisplays  the  pilot's  single  task  load  values  and  sets  the  configuration  of 
*hwl*  to  'configl 


NAME:  DISPLA Y-CPG-SINGLE-TASKS 

DESCRIPTION:  calls  all  the  functions  necessary  to  display  the  cpg's  single  task  load 
values  and  sets  the  configuration  of  *hw  1 * to  'config2 
FUNCTIONS  CALLED:  DISPLAY-CPG-SINGLE- TASK-LOADS 

NAME:  REDISPLA Y-CPG-SINGLE-TASKS 

DESCRIPTION:  redisplay’s  the  cpg’s  single  task  load  valuesand  sets  the  configuration  of 
*hwl*  to  'config2 


NAME:  PILOT-TASK-COMBINATIONS 

DESCRIPTION:  calls  all  the  functions  necessary  to  display  the  pilot's  combined  task  load 
values  and  sets  the  configuration  of  *hwl*  to  'config3 

FUNCTIONS  CALLED:  GET-AND-DISPLAY-PILOT-COMBINED-TASK-LOADS 
CALLING  FUNCTIONS:  START-PILOT-PROCESS 


NAME:  REDISPLAY-PILOT-TASK-COMBINATIONS 
DESCRIPTION:  redisplay’s  the  pilot's  combined  task  load  values  and  sets  the 
configuration  of  *hwl*  to  ’config3 


NAME:  CPG-TASK-COMBINATIONS 

DESCRIPTION:  calls  all  the  functions  necessary  to  display  the  cpg's  combined  task  load 
values  and  sets  the  configuration  of  *h w 1 * to  ’config4 
FUNCTIONS  CALLED:  GET-AND-DISPLAY- CPG -COMBINED-TASK-LOADS 
CALLING  FUNCTIONS:  START-CPG-PROCESS 


NAME:  REDISPLA Y-CPG-TASK-COMBINATIONS 
DESCRIPTION:  redisplay’s  the  cpg's  combined  task  load  values  and  sets  the 
configuration  of  *hwl*  to  'config4 

NAME:  START-PILOT-PROCESS 
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DESCRIPTION:  starts  the  process  to  get  the  list  of  pilot  active  tasks  from  the  active  task 
FUNCTIONS  CALLED:  PILOT-TASK-COMBINATIONS 


DBO(lnO»SS'S'iS  .o  get  me  Us.  of  cpg  active  tasks  from  the  active  task  Us, 
Actions  called:  cpg-task-combinations 

a xlear-history  message  to  clear  the  pilots > task  load 
SfcSSon  d£  from  window-panes  in  task-load-window-frame:  conftgl  and 
’config3 

DEsSlSFo  tds'fu^tion  sends  a xlear-history  message  to  clear  the  cpgs  task load 
^d  cSSon  datt  from  window-panes  in  task-load-window-frame:  config2  and 

'config4 

5 2 4 FG4:  The  task-load  formatting  functional  group 


FILE:  SQUID:>stav>midas-tlm>tlm-Ioad-Iists.lisp 

DESCRIPTION:  This  file  contains  the  functions  and  flavors  necessary  to  construct  the 
^nnONS DEF^aGE^^TC-MOTOR-ELEMENT-LIST  GENERATE- 

gsss-"” 

Ivors' dSId:^ load^ls  motor-component  cognitive- 

COMPONENT  AUDITORY-COMPONENT  VISUAL-COMPONENT  TASK 
NAME  OF  THE  MACRO:  PUSH-END 

DESCRIPTION:  adds  list  to  the  end  of  another  list 
INPUTS:  VALUE  VALUE-LIST-PLACE 

NAME:  SET-INST-V ARS-FROM-LIST 

DESCRIPTION:  Utility  function  for  setting  instance  variables. 

CAU^b^PUNCTI^S^ASS  WN^I^T- V ARS-ELEMENTS- AND-LOADS 

DESCRIPTIONt  defines  a task  as  an  instance  of  each  component,  each  with  iwo  instance 

VmW^TA^^VlSUAL  VI  INDEX-V  N0TV1  AUDITORY  Al 
EfScU  Cl  INDEX-C  NOTC1  MOTOR  Ml  INDEX-M 

^fETHODS  UPDATE-MOTOR  UPDATE-COGNITIVE  y]S 

UTO ATE-visU AL  DO-MTR- MAGIC  DO-COG-MAGIC  DO-AUD-MAOIC  DO-VIS- 

MAGIC 
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NAME:  VISUAL-COMPONENT 

ggg™:«6p*  elements  of  the  visual  dimension  and  sets  them  to  nil 

VARIABLES:  NEAR  FAR  SCAN  FIXATE  INTEGRAL  SFPARAmc 
OBJECTS  FEATURES  SALIENT  MASKED  STATIC  DYNAMIC 

NAME:  AUDITORY-COMPONENT 

*5  e^,lnts  of  ^ auditory  dimension  and  sets  them  to  nil 
MASKED^  VARIABLES:  orIENT  DISCRIMINATE  SYGNAL  SPEECH  SALIENT 

NAME:  COGNITIVE-COMPONENT 

™ ei5ments  of  cognitive  dimension  and  sets  them  to  nil 

SPAT^LA^D  W?SdTRANSF°RM  SIN0LE  MULTIPLE 

NAME:  MOTOR-COMPONENT 

rightTe^SbothSS  FlNE  M0UTH  HEAD  EYE  FEETFINGER 
NAME:  LOAD-VALS 

COGNITIVE  WITHIN-MOTOR  VISUAL-AUDITORYAUDnSRY 

M^C0GNmVE  ^VE-MOTORTOUttZSS^ 
nF^^^?nMN'JNS7'VARS'ELE^ffiNTS'AND'L0ADS 

S2  ! C"0n  ““  elemtm  list  va]u's  “J  >«iens  them  to  their  respective 
INPUTS:  ELEMENT-LIST  LOAD-LIST 
FUNCTIONS  CALLED:  SET-INST-VARS-FROM-LIST 
CALLING  FUNCTIONS:  MAKE-TASK-LOAD-LIST  SHOW-TASK-I  DAn 
DISPLAY-SINGLE-TASK-LOADS  DISPLAY-COMBINED-TASK-LOADS 

NAME:  SET-INST-VARS-FROM-INDICES 

i?adSr^S°N:  *“*  aSSignS  thC  indiceS  l°  ** instance  variables  that  the  function  generate 

INPUTS:  TASK-INDEX-LIST 
OUTPUTS:  TASK-OBJECT 

CALLING  FUNCTIONS:  SHOW-TASK-LOAD  DISPLAY-SINGI  F tacit  i r\ am 

^S.ALBf°MBINEDTASK'L0ADS^ 

NAME:  SUM-VALS-ACROSS-LISTS 

DESCRIPTION:  sums  across  each  element  in  each  list  for  each  task  in  a task  combination 

dm:s  «*  element  was  selected  eombinaoon 

INPUTS:  LIST-OF-ELEMENT-VALUES 
OUTPUTS:  LIST-OF-ELEMENT-VALUES 

UMSFMIXTLM^SkTom>TASK  L0AD  DISPLAY'C0mbined-TASK- 
NAME:  GENERATE-ELEMENT-LIST 
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^ C’e",en‘S  ^ 

INPUTS:  TASK  tmctampp  VARIABLE  INDEX-V  TASK  INDEX-V) 

FUNCTIONS  CALLED:  g^^^fXA^^^S^l^VARIABLE  INDEX-A 

*^sSss&« 

GENERATE-MOTOR- 

the  list  of  indices  used  to  access  the  vsual  tnatnx 
INPUTS:  COMPONENT-INDICES 

?!SmGSroN^ON^OT5SlATE.ELEMENT-LIST 

SKKScC.  the  Us.  of  indices  used  to  access  the  authtoty  tnatnx 
INPUTS:  COMPONENT-INDICES 

OUTPUTS:  AUDITORY-ELEMENTS  T 

CALLING  FUNCTIONS:  GENERATE-ELEMENT-LIS 

SF^SS^CTeS^aT^lfe1 Se  cognitive  component  that  were 

“SSSS%e.  used  to  access  the  cognmve  tnatnx 
INPUTS:  COMPONENT-INDICES 

SS^OT^GM^ELEMEKr-LIST 

**  COmp°"?n'  ^ WCrc 

STJSKhsiof  Mces  used  to  access  the moan  tnantx 
INPUTS:  COMPONENT-INDICES 
OUTPUTS:  MOTOR-ELEMENTS  ^ , TQT 

CALLING  FUNCTIONS:  GENERATE-ELEMENT-L 

FILE:  SQUID:>stav>midas-tlm>tlm-display-loads.lisp 

DESCRIPTION:  This  file  contains  the  functions  that  take  lists  of  task  element  and  o 

DISPLAY-PILOT^OMBMD-TASK-LOADS  ^SPL  letask 

1 -°%l  MAKE-TASK-LOAD-L.ST 

NAME:  *TASK-NUMBERS* 

INITIAL-VALUE:  nil 
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DESCRIPTION: 

NAME:  MAKE-TASK-LOAD-LIST 

them  to  the  functions  Aa^genSteUie^k  dSficatiom^d  S thCm  f"  8 list  passes 

ou-mrrsNUMBER'°F’TASKS  &0PTI0NAL  ust-of-task-objects 

NAME:  SCHEDULER-LOAD-LIST 

SXSjf  Renuka  funcdu"  10  Of  loading  values  for 

INPUTS:  LISTS-OF-TASK-INDICES 

FUNCTIONS  CALLED:  SET-INST-VARS-FROM  INDICES  GENERATE-LOAD 
NAME:  SHOW-TASK-LOAD 

M,a  mudei  “ ■“  ‘oca,“ia«  **  <*■** 

INPUTS:  LISTS-OF-TASK-INDICES 
NAME:  DISPLAY-SINGLE-TASK-LOADS 

displays  th7S^S^  Ioad  and  element  values  “d 

isisr^ 

PRINT"TO'  fASK-ELEMENT-AND-LOAD-PANE 
SmGLEN?A^NSADT;  DISPLAY-PILOT-S1NGLE-TASK-L0ADS  DISPLAY-CPG- 

DFS^R>^omAY'PIL,?T'SINGLE-TAS^  LOADS 
^calculates  theffad  and^memlSS  S^E 

S^GlStASICLOADs'  GET'PILOT'TASK  i Nl  n - r s-FROM-TABLE  DISPLAY- 
CALLING FUNCTIONS:  DISPLAY-PILOT -SINt  .LE-TASKS 

NAME:  DISPLAY-CPG-SINGLE-TASK-LOADS 
CALLING  FUNCTIONS:  DISPLAY-CPG  SINGLE-TASKS 
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KTA\yTP*  nVT- A ND-DISPLAY-PILOT-COMB1NED-T ASK-LOADS 
DESCRIPTION*  gets  all  the  pilot  single  task  indices  from  the  task-load-data-hash-mble 
£d  akSZ \heirgcoIined"oad  and8element  lists  and  then  outputs  the  pilot  combined 

task  values  to  the  pilot  task  combination  load  pane  nn  rw  Amvp  TASKS 

FUNCTIONS  CALLED:  UPDATE-SYM-TO-JILL  GET-PILOT-ACTIVE-TASKS 

tad  mi  element  lists  and  Uten  outputs  the  cpg  eombmed  task 

values  to  the  cpg  task  combination  load  pane  A r^n\rc  ta  c 

'function S CALLED : UPDATE-SYM-TO-JILL  GET-CPG-ACTIVE-TASKS 

DISPLAY-COMBINED-TASK-LOADS 
CALLING  FUNCTIONS:  CPG-TASK-COMBINATIONS 

NAME:  DISPLAY-COMBINED-TASK-LOADS  , 

DESCRIPTION:  takes  a list  of  index  lists,  calculates  their  load  and  element  values 

displays  the  load  and  element  values  for  single  t^k  TASK  NAME- 

INPUTS:  CURRENT-TIME  LISTS-OF-TASK-INDICES  LIST-OF-T A S K-NAMt 

mSTwrrtn\iS  PAI I FD-  SET-INST-VARS-FROM-INDICES  GENERATE- 

S£Sv£5oss.lists  GENERATE-LOAD  ASSIGN-INST- 

Pc™“o^Do“ 

GET-AND-DISPLAY-CPG-COMBINED-TASK-LOADS 

DESOiPTTOICTWs  Function  generates  instances  of  tasks,  puts  them  in  a list  and  passes 
them  to  the  functions  that  generate  the  task  classifications  and  evaluate  each  task  and  tas 
combination  and  returns  two  lists:  element  and  load  values,  that  are  used  to  initialize  the 

vacm  component  and  load  instance  variables  that  get  displayed  txtt^t/^cc 

tmS] itcT NTT i M R FR  OF-TASKS  &0PT10NAL  LISTS-OF-TASK-INDICES 
RINOTO?ScALLm  CREATE “TASK TASK-COMPONENTS  GENERATE- 
ELEMENT-LIST  GENERATE-LOAD  SET-INST-VARS-FROM-INDICES  SUM- 
VALS-ACROSS-LISTS 

CALLING  FUNCTIONS:  MAKE-TASK-LOAD-LIST 
5.2.5  FG5:  The  midas-interface  functional  group 

FILE:  SQUID:>stav>midas-tlm>test-midas-tlm.lisp 

DESCRIPTION:  This  file  contains  one  function  used  solely  for  testing  the  tlm  and  for 
hard-coding  the  tasks  used  in  the  simulation 
FUNCTIONS  DEFINED:  GENERATE-TASK-LIST 

DESOTffTO^this  function  is  for  testing  the  tlm  and  for  demos.  It  creates  a list  of  active 
tasks  that  are  evaluated  and  displayed. 

OUTPUTS:  TTME-AND-TASK-LIST 


Page  C-47 


FILE:  SQUID:>stav>inidas-tlm>sym-to-jill.lisp 

DESCRIPTION:  This  file  contains  the  parameter  whose  value  is  the  list  of  tasks  that  the 
TLM  processes 

PARAMETERS  DEFINED:  TASK-COMBINATION-LIST* 

NAME:  *TASK-COMBINATION-LIST* 

DESCRIPTION:  assigns  a list  of  tasks  to  *task-combination-Iist*  that  acts  as  input  to  the 
TLM  r 

INITIAL-VALUE:  GENERATE-TASK-LIST 


FILE:  SQUID:>stav>midas-tlm>current-sym-to-jill.lisp 

DESCRIPTION:  This  file  is  to  be  used  to  recieve  the  active  task  list  update  eveiy  ten  ticks 
from  the  mission  decomposition  module.  However,  it  is  not  in  use  because  it  was  never 
linked  to  up  and  tested. 


FILE:  SQUID:>stav>midas-tlm>sym-interface.lisp 
Interface  Files  are  specified  in  "B:>Midas>gIobals>interface-globals.lisp" 

AUTHOR:  Jerry  Murray 

DESCRIPTION:  This  file  contains  the  functions  necessary  to  link  the  task  loading  model 
and  the  task  decomposition  model  so  that  the  simulated  tasks  can  be  evaluated  by  the  task 

vadAd^'  r^x^kwas  CTeated  by  Jerry  Murray  and  modified  by  Jerry  Murray. 
VARIABLES  DEFINED:  *INTERFACE-FILES*  *TASK-COMBINATION-LIST* 
JUNCTIONS  DEFINED:  UPDATE-Z-TO-JILL  UPDATE-JILL/TO-Z  UFDATB-S  YM- 
UPDATE-Z-TO-SYM  UPDATE-SYM-TO-JILL  UPDATE- Jn  J -TQ-gYM  INIT- 

fN™ji^TO-SY'lML'TaZ  INIT-SYM  T0  Z INIT-Z-TaSYM  INIT-SYM-TO-JILL 


NAME:  *INTERFACE-FILES* 

DESCRIPTION:  the  value  of  this  variable  is  set  to  the  different  files  necessaiy  to  link  the 
dm  and  mission  decomp  model  3 

INITIAL-VALUE: 

(JILL-TO-SYM  B:>Midas>interface>files>Jill-to-sym.lisp 
SYM-TO-JILL  squid:>stav>midas-dm>sym-to-Jill.lisp 
Z-TO-SYM  B:>Midas>interface>files>Z-to-sym.lisp 
S YM-TO-Z  B :>Midas>interface>files>sym-to-Z.lisp 
JILL-TO-Z  B:>Midas>interface>files>Jill-to-Z.lisp 
Z-TO-JILL  B:>Midas>interface>files>Z-to-Jill.lisp) 

NAME:  INIT-SYM-TO-JILL 

.DESCRIPTION:  This  function  stuffs  the  value  of  *Interface-Files*  :cuirent-sym-to-Jill 

into  ^Interface-Files*  :sym-to-Jill 

CALLING  FUNCTIONS:  MIDAS-TLM-DISPLAY 


NAME:  UPDATE-SYM-TO-JILL 

DESCRIPTION:  this  function  reads  *Interface-Files*  for  the  value  of  :current-svm-to-JiU 
^id  checks  :sym-to-Jill  to  see  if  its  the  current  version,  if  not  it  then  stuffs  ♦Interface- 
Files*  :current-sym-to-Jill  into  *Interface-Fi!es*  :sym-to-Jill  and  returns  *task- 
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combination-list*'  which  is  the  value  of  -Interface-Files*  :sym-to-Jill  as  a hard-coated  value 

'cSllJNcfFUNCTlOhfi^^CT-AJSj'DISPLAY-PILXyr-COMBINHD-TASK-LOADS 

GET-AND-DISPLAY-CPG-COMBINED-T  ASK-LOADS 


NAME:  INTT-JILL-TO-SYM 
DESCRIPTION:  Not  used  by  the  task  load  model 


NAME:  INIT-Z-TO-SYM 

DESCRIPTION:  Not  used  by  the  task  load  model 


NAME:  INTT-SYM-TO-Z 

DESCRIPTION:  Not  used  by  the  task  load  model 


NAME:  INIT-JILL-TO-Z 

DESCRIPTION:  Not  used  by  the  task  load  model 


NAME:  INTT-Z-TO-JILL 

DESCRIPTION:  Not  used  by  the  task  load  model 

NAME:  UPDATE-  JILL-TO-SYM 
DESCRIPTION:  Not  used  by  the  task  load  model 

NAME:  UPDATE-Z-TO-SYM 
DESCRIPTION:  Not  used  by  the  task  load  model 

NAME:  UPDATE-SYM-TO-Z 
DESCRIPTION:  Not  used  by  the  task  load  model 

NAME:  UPDATE- JILL-TO-Z 
DESCRIPTION:  Not  used  by  the  task  load  model 


NAME:  UPDATE-Z-TO- JILL 
DESCRIPTION:  Not  used  by  the  task  load  model 


5.2.6  FG6:  The  system-initialization  functional  group 

FILE:  SQUID:>stav>midas-tlm>tIm-files-init.lisp 

DESCRIPTION:  This  file  initializes  and  opens  the  task  load  window  frame  object,  creates 
the  hash-tables  of  the  task  name  and  index  associations,  creates  the  dm  process  and  binds  it 

to  the  H key,  resets  the  displays  .uw.<l 

VARIABLES  DEFINED:  -TLM-PROCESS* 

FUNCTIONS  DEFINED:  MIDAS-TLM-DISPLAY  RESET-TLM 


DESCRIOTION:  This  function  sets  the  task-load-window-frame  object  - -hwl*  - to  an 

arbitrary  value  , , , . , - 

INITIAL- VALUE:  tv:make-window  'task-load-window-trame 

: save-bits  nil 
:expose-p  nil 

NAME:  PELOT-TASK-LOAD-DATA-TABLE  . u 

DESCRIPTION:  This  function  call  creates  the  pilot-task-load-data-table  hash  table 
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NAME:  CPG-TASK-LOAD-DATA-TABLE 

DESCRIPTION:  This  function  call  creates  the  cpg-task-load-data-table  hash  table 
NAME:  CPG-SHORT-NAMES-TABLE 

DESCRIPTION:  This  function  call  creates  the  init-short-names-table  hash  table 
NAME:  PDLOT-SHORT-NAMES-TABLE 

DESCRIPTION:  This  function  call  creates  the  init-short-names-table  hash  table 
NAME:  MIDAS-TLM-DISPLAY 

DESCRIPTION:  This  function  opens  the  midas-tlm  window  and  starts  the  command  menu 
and  read  tile  processes 

FUNCTIONS  CALLED:  INTT-SYM-TO  JILL  PiLOT-TASK-DISPLAY  PG-TASK- 
DISPLAYRESET-TLM-DISPLAY 

METHODS  CALLED:  (METHOD  PRINJ-THE-1N DIAL-DISPLAY  TASK-LOAD- 
WINDOW-FRAME) 

NAME:  *TLM- PROCESS  * 

DESCRIP1  ION:  This  sets  the  variable  * 1 L;vl  PROCESS*  to  the  midas-tlm  process  that 
runs  off  of  the  "H"  key  when  loaded 

INITIAL- VALUE:  (UNLESS  (AND  (VARIABLE  BOUNDP  *TLM-PROCESS*) 

(TYPEP  TLM-PR  OCESS  * ’PROCESS) 

(STRING-EQUAL  (PROCESS-NAME  *TLM-PROCESS*) 
Task  I.xrad  Model)) 

(PROCESS-RUN-FUNCTION  Task  Load  Model 
•MIDAS-TLM-DISPLAY)) 


NAME:  tv:add-select-key 

ARGUMENTS:  #\H  task-load-window-trame  ' Task  * ,oad  Window  Frame" 

INITIAL  VALUE:  nil 

DESCRIPT  ION:  This  function  calls  the  muias-tlrn  as  a process  that  runs  off  of  the  "H” 
key  when  loaded 


NAME:  RESET-TLM 

DESCRIPTION:  this  function  resets  the  variable  TLM-PROCESS*  to  the  midas-tlm 
process  and  the  dm  interface  to  the  initial  configuration:  no  loads  or  tasks  are  disnlaved 
FUNCTIONS  CALLED:  CLEAR-PILOT-PANES  Cl  EAR-CPG-PANES  X 


5.2.7  FG7:  The  tlm-system  functional  group 

FILE:  SQUID:>stav>midas-tIm>midas  tlm.il  p 
DESCRIPTION:  This  file  creates  the  system  - .Mil.) AS  ULM. 


SYSTEM  DEFINED:  MIDAS-TLM 
(defsystent  MIDAS-TLM 
(:pretty-name  "MIDAS  Task  Loading  Model 
:short-name  "tlm" 

.'default-pathname  "midas-tlm:midas-tlm:  ) 
(:  serial 

"task-load-interface-frame" 

"tlm-load-lists" 

^parallel 
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"tlm-editor" 

"tlm-calculator" 

" tlm-task-  load-data 

"tlm-display-loads” 

"task-names" 

"select-task-display" 

"test-midas-tlm" 

"sym-interface" 

"current- sym-to-jill" 

"sym-to-jill") 

"tlm-files-init")) 


6.0  USER’S  GUIDE 

machine  and  provide  the  required  input. 

6.1  OVERVIEW  OF  PURPOSE  AND  FUNCTIONS 

The  design  team  members  interact  ^^JVwIS5cs?SmaSine.  These  menus 
momentary  pop-up  menus  on  the  nomtor  of  for  singie-tasks  or 

XSSSS 

The  loading  values  of  ™the  raskW.  and  the  lime 

o^^ndin^o  when^he’tas'k  Vrfomance^as  simidaw^^ie  dwigi^  task 

display  horizontally  and  veracally  tc ■ dtsp  ay  J^ffarTte  left  margin  scrolls 
classification  and  loading  profiles.  SclagssTfication  data.  Pointing  at  a scroll  bar  in  the 
SSf -kToadi  classif, canon  data  a,  each  rime  or  rick  increment  ,n 

the  simulation* 

The  designer  has  the  option  to  stop  the  M1DAS-TLM  process  at  any  time,  clear  the  displays 
and  restart  the  process. 

The  designer  is  currently  re££ct?f  ™ rimulati thetasks  generated  by 
and  coded  as  a hash  table.  This  is  ^Pre  classified  and  coded  before  a simulation, 

but  onM^e^f  name^d^ndexe^^coded^as  haslf^tles,  only  the  names  of  the  tasks 
need  t^be  passed  to  the  TLM  fot  task  evaluatton. 


INSTALLATION  AND  INITIALIZATION 


Jrn^^ 

MIDAS -TLM  system  is  now  ready  to  be  loaded  mio  u 

The  MIDAS-TLM  is  installed  as  a system  in 

logs  on  ha  squid  and  comptles  the  midas-rim  system 
fiSm  the  dynamic  Lisp  listener  window  by  typing  the  command 
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Compile  System  Midas-TIm. 

user  ^SSPSe“.S-H0adS  *'  SyS"m'  T°  Xka  **  M^AS-IUrf  display.  Ihe 

Edit  System  Midas-TIm 
6.3  STARTUP  AND  TERMINATION 

SSS 

moment  ^'SSrSh^SSS^' ^To™W„<S.^1&1IS5  "f 1 5 V 
SSSon.  oS«  S ^l^^^SS^Si'iih“r?ch  Sck:incren,'nt 

r»c£“iates  Md  ***•  - “s  -< 

"Tnnf^/r115  the  JLM-process  by  clicking  the  mouse  on  either  the  "Pilot"  or 
men?  wl  SaMs™^  B • ">omenuuy 

bottom  and  left-Sid^dS  Sf  Ksplay-  “*  *"*  ClaSsifications  via  50011  ***  on  the 

“alyse?  Sa,Uendal'>’  flayed  and  scrolled  through  forfost  Jmulatta, 

Sf  * lhe  mOU?  °". *>»  "Cnnw 

TLM,  at  which  point  the  user  t«  be/n  ag?in  P * TLM-process  and  re-imtializes  the 
The  user  follows  the  same  procedures  to  run  the  TLM  during  each  simulation. 
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data 


6.4  FUNCTIONS  AND  THEIR  OPERATION 

There  are  four  flies  conlaiuiug  the  fuuetions  that  control  <•«  "d  c°"c,i,'  **  "*k 

tSles  These  ‘wo  flies  need  to  be  edited  to  ron  the  TLM  dunng  simulations. 

SsSSSSSSSSSir 

processes,  and  call  functions  to  clear  or  reset  the  displays, 
from  the  window-panes. 

classification  data  into  these  tables  and  compile  the  buffer. 

ssssasass^^g® 

must  also  be  edited,  replacing  the  current  names  with  the  ones  that  will 
simulation.  This  buffer  must  also  be  compiled. 

now  ready  for  simulation. 

6.5  ERROR  AND  WARNING  MESSAGES 

The  nossible  error  messages  that  may  occur  would  probably  be  associated  with  either 

1 “aah  tables  or 

that  aren't  in  the  hash  tables. 


Page  C-53 


6.6  RECOVERY  STEPS 


Recovery  from  any  error  states  is  accomplished  by  clicking  on  the  "RESET-TLM"  menu 
item  in  the  pop-up  display  associated  with  the  "Crew  Selection  Menu"  display  label. 


7.0  ABBREVIATIONS  AND  ACRONYMS 


ACbw 
Ac  w 

CAD 

Cb 

CCSCI 

CCw 

CMbw 

CPU 

CSCI 

CTA 

Cw 

FG1: 

FG2: 

FG3: 

FG4: 

FG5: 

FG6: 

FG7: 

I/O 

Lc 

Lisp 

LMFS 

Lv 

Mcw 

MIDAS 

SCI 

SSCI 

TACSCI 

TDSCI 

TECSCI 

TLM 

TLMSC 

TLMSCI 

UICSCI 

VACM 

VACP 

VCbw 

^ C H 


Auditory-Cognitive  between  matrix 
Auditory  within  matrix 
Computer  Aided  Design 

Conflict  Value  for  between  matrices 
CalculatorComponent  Software  Configuration  Item 
Cognitive  within  matrix 

Cognitive-Motor  between  matrix 
Central  Processing  Unit 
Component  Software  Configuration  liem 
Cognitive  Task  Analysis 

Conflict  Value  for  within  matrices 
Functional  Group  1:  The  task  editor  and  data-input 
Functional  Group  2:  task-load  calculator 
Functional  Group  3:  display  windows  and  menus 
Functional  Group  4:  task-load  formatting 
Functional  Group  5:  midas-interface 
Functional  Group  6:  system-initialization 
Functional  Group  7:  tlm-system 
Input/Output 

Auditory  load  value 
Cognitive  load  value 

List  processor  (lots  of  silly  irritating  parentheses) 

Motor  load  value 

Lisp  Machine  File  System 
Visual  load  value 
Motor  load  value 

Man-Machine  Design  and  Analysis  System 
Software  Configuration  Item 
Scheduler  Software  Configuration  Item 
Task  Adjustor  Component  Software  Configuration  Item 
Task  Decomposition  Software  Configuration  Item 
Task  Editor  Component  Software  Configuration  Item 
Task  Load  Model 

Task  Load  Model  Software  Component 

Task  Load  Model  Software  Configuration  Item 

User  Interface  Component  Software  Configuration  Item 

Visual-Auditory  between-mairix 

Visual  Auditory  Cognitive  Motor 

Visual  Auditory  Cognitive  PsychoMotor 

Visual-Cognitive  between-matrix 

Visual  within-matrix 
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8.0  GLOSSARY 

Abstraction  Hierarchy  A hierarchical  set  of  invariant  qualities  that  are  common  to  the 
the  human  information  processing  systems  capability  to  represent  and  manipulate 

Additional-Cost  An  unknown  function  or  constant  that  accounts  for  the  obtained 

increases  in  the  magnitude  of  workload  predicted  by  an  averaging  model.  . . 

Attention  A mental  control  mechanism  that  guides,  focuses,  or  elaborates  the  acquisition 
and  processing  of  information. 

Auditory  A modality  of  perception  involving  aural  stimuli. 

Auditory-Cognitive  The  interaction  of  aural  perceiving  and  central  processing 

Auditory-Visual  The  interaction  of  aural  perceiving  and  visual  perceiving. 

Averaging  Model  A model  that  predicts  retrospective  workload  ratings  by  averaging  the 
difficulty  of  the  events  or  stimuli  that  are  experienced.  . 

Behavioral  State  A description  of  the  attributes  and  values  that  constitute  a specific 

pattern  of  human  performance.  , . , r . „ 

Best-Fitting  Model  A model  that  best  describes  the  obtained  patterns  of  human 
nerformance  or  behavioral  states.  , , ^ 

Between-Matrix  A two  dimensional  matrix  of  values  that  represents  the  demand 

incurred  on  the  human  information  processing  system  from  the  interaction  of  information 

processing  mechanisms  in  different  stages.  ...  , 

Bottom-Up  Model  A human  performance  model  that  is  dictated  by  a detailed  set  of 

primitive  elements  of  human  behavior.  . . 

Cognitive  The  central  processing  stage  composed  of  at  least  the  attentional, 
transformational  and  memory  mechanisms  and  processes  that  act  on  perceived 
information,  and  which  prepare  the  information  processing  system  to  generate  a response 

Cognitive  Task  Analysis  An  analytic  technique  to  explicate  the  interactions  among  the 
current  and  desired  states  of  the  world  and  agent,  and  the  representations  available  to  the 
agent  that  are  necessary  to  model  human-system  interactions. 

Cognitive-Motor  The  interaction  of  central  processing  mechanisms  and  motor  response 

Component  Software  Configuration  Item  A component  of  a software  architectural 
unit/item  used  to  implement  a specific  model  or  tool  within  MIDAS.  , . 

Conflict  Matrix  A two  dimensional  matrix  of  values  that  represents  the  demands  and 
conflicts  among  the  structural  and  procedural  psychological  attributes  that  describe  a tas  . 
Conflict  Value  A specific  value  that  represents  the  interacnon  between  two  structural  or 

procedural  psychological  attributes  classifying  a task  tuat 

Dimension  A top  level  set  of  structural  or  procedural  psychological  attributes  that 
corresponds  to  a specific  stage  of  information  processing.  This  is  the  level  that  load  values 

Element  A bottom  level  structural  or  procedural  psychological  attribute  that  is  used  to 

Event  Base?  A simulation  that  progresses  according  to  changes  in  state  variables  that 
represent  specific  events  that  occur  in  the  environment  or  behavior  of  the  operator. 

Genera  7.2  The  symbolics  Lisp  operating  system. 

Human  Information  Processing  The  human  mental  activities  and  structures  that 
represent  and  manipulate  information  symbolically,  and  which  enable  humans  to  perceive 
and  respond  to  changes  in  external  (environmental)  or  internal  (mental)  states. 
HumanPerformance  Model  A quantitative  (analytic  or  computer-based)  representation 
or  description  of  all  or  parts  of  human  operators  or  maintainers  of  complex,  dynamic 
systems. 
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Invariant  Property  A characteristic  of  the  information  processing  system  that  does  not 
change  as  a function  of  the  information  extracted  from  the  perceptual  array. 

Load  Balancing  Strategy  A behavioral  change  in  an  operator  as  a function  of  the 
imposed  visual,  auditory,  cognitive  and  motor  loads  imposed  by  performing  a task.  The 
effect  of  the  behavioral  change  is  a regression  to  the  mean  woridoad  (reduction  of  peak 
loads)  of  the  pertinent  tasks  with  regard  to  alloted  time. 

Load  Value  A value  that  represents  the  amount  of  psychological  resources  required  to 
perform  a task  relative  to  performing  another  task  or  set  of  tasks. 

Memory  The  psychological  mechanisms  that  maintain  information  over  time. 

Mental  Workload  An  evaluation  about  the  difficulty  of  ongoing  experiences  and  the 
impact  of  those  experiences  on  the  physical  and  mental  states  of  an  operator.  The  evaluation 
is  a function  of  the  collection  of  attributes  that  may  or  may  not  be  relevant  in  controlling  the 
evaluations  or  behavior  that  depend  on  the  circumstances  and  design  of  a given  task(s)  and 
the  a priori  bias  of  the  operator. 

Motor  The  effector  mechanisms  of  the  human  body. 

Multi-Task  Model  A comprehensive,  quantitative  model  that  represents  the  combined 
effects  of  a variety  of  tasks  on  human  performance. 

Object-Oriented  Programming  Programming  languages  that  represent  data  structures 
as  objects  with  attributes  and  values. 

Output  Model  A model  that  focuses  on  the  result  of  human  performance. 

Normative  Model  A model  that  predicts  how  an  operator  should  perform  by  assuming 
rational  operator  behavior.  6 

Phase  IV  The  period  of  research  and  development  on  MIDAS  from  the  small  offsite  in 
March  1989  to  the  end  of  demos  in  July  1990. 

Phase  V The  period  of  research  and  development  on  MIDAS  from  the  large  offsite  in 
November  1990  to  the  end  of  demos  sometime  in  late  1991  or  early  1992. 

Physical  System  The  design  of  the  system  hardware  and  software. 

Process  A specific  information  processing  mechanism  that  manipulates  information 
symbolically. 

Resource  The  attentional,  physical  and  memory  capabilities  of  an  operator. 

Scheduling  Model  The  software  configuration  item  that  sequences  the  order  of  the 
simulated  tasks. 

Constraints  The  imposed  order  of  interactions  between  the  dimensions  used  in 
the  TLM. 

Simulation  Executive  The  software  configuration  item  used  to  control  the  simulation 
by  controlling  the  flow  of  execution  of  the  rest  of  MIDAS'  software  configuration  items. 
Software  Component  A component  of  a software  architectural  unit/item  used  to 

implement  a specific  model  or  tool  within  MIDAS. 

Software  Configuration  Item  An  architectural  unit  of  software  used  to  implement  a 
specific  model  or  tool  within  MIDAS.  v 

fymbol^all^  sPec^lc  information  processing  mechanism  that  represents  information 

j“?inyn8  Model  A model  that  predicts  retrospective  workload  ratings  by  summing  the 
difficulty  of  the  events  or  stimuli  that  are  experienced. 

Task  Decomposition  Model  The  software  configuration  item  that  decomposes  high 
level  goals  into  the  simulated  tasks. 

Task  Loading  Model  The  software  configuration  item  that  predicts  the  visual,  auditory 
cognitive  and  motor  loads  that  the  simulated  tasks  impose  on  the  operator  of  the  system. 

Tick  Based  A simulation  that  progresses  according  to  changes  in  state  variables  that 
represent  specific  events  that  occur  in  the  environment  or  behavior  of  the  operator 

User  A g0°?  question  and  currentJy  ill-defined,  but  usually  meant  to  be  an  operator  of  the 
MIDAS  workstation. 


Variant  Property  A characteristic  of  the  information  processing  system  that 
function  of  the  information  extracted  from  the  perceptual  array. 


changes  as  a 
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Visual  A modality  of  percepUOT  nerwivbgUMd  central  processing 

Visual-Cognitive  The  interacuon  of  visual  perceiving  *u 

mechanisms.  . f values  that  represents  the  demands  incurred 

sbsss—s— ^ *•  *— ■*■ of  mfommrion  proc'ssm8 

mechanisms  within  the  same  stage. 

9.0  NOTES 

The  software  contained  in  this  The  software 

before  phase  IV , and  was  completed  Hpvplnned  in  that  four  month  period,  and  was 

SIS"  W demos.^This  software  wii,  provide  *e  core  for 

future  TLM  software  development  efforts. 

9.1  LIMITATIONS 

Tie  limhadonsofrheTlMSCI  rue  primarU, 

nudel-  The  model  is unresred and  so  vihMy  Jft? tack of  uXfflnding 

structure  of  the  model  may  not  be  the  most  optimal ^a  resuu  . o ^ hs 

of  the  models  strengths  and  weaknesses,  notions  of  the  most  appropriate 

algorithms.  This  lack  of  understanding  leads  to  3^0“jinexpericncS  and  even 
future  computational  efJort%^lt£  j £ gui?  the  efficiency  aS  efficacy  of  the 
SSESSZSS&tt  Ef5W4  and  may  lead  to  problems  in  furore 
development  efforts. 

9.2  LESSONS  LEARNED 

lcar^jg  how^^o^ai^  hw  *C 

meaning  of  AI,  the  author  learned  far  more  than  he  intended. 

9.3  FUTURE  DIRECTIONS 

The  future  software  development  to  the  TLM  by 

Task  Adjustor  TLM  component.  This  co  ipo  ssifications  in  order  to  tailor  the  tasks  to  the 
providig  the  mechanisms  to  adjust  the  “ c wiU  allow  ^ tasks  to  be  automatically 

the  current  operating  environment.  This  cap  Operator  Model.  This  capability 

MIDAS-TLM  system  for  use  in  simulations. 

values  available  for  .he  TLM  ro  class, fy  tasks. 
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(OR=orient,  Discriminate,  Signal.  SP=spatial,  SA=salient,  MA=masked). 
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FIGURE  A-3  CONFLICT  MATRIX  FOR  THE  MOTOR  DIMENSION 


(VE  verbal,  SP  spatial,  NE— near,  FA— far.  Di-discrete,  CO=continuous,  GR=gross 
FI=fine,  MO=mouth,  HE=head,  EY=eye,  HA-hand,  FE=feet,  FI=finger,  RI=neht  ’ 

LE=left,  BO=both).  ’ 
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DIMENSION 


VISUAL  COGNITIVE 


(DI=dir«t  Transformation,  SC=single  choice,  MOmultiple  choice,  VE-vctbal 
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mam  MACHINE  INTEGRATION  DESIGN  & ANALYSIS  SYSTEM 
(MWAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 
( / PHASE  IV: 

SYMBOLIC  EQUIPMENT  MODEL 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 

This  document  gives  the  requirements  and  design  of  the  Equipment  Model  Software 
Component  for  MIDAS  Phase  IV. 


1.2  PURPOSE  OF  DOCUMENT 

object-oriented  systems  is  assumed. 


2.0 


RELATED  DOCUMENTATION 


2.1  APPLICABLE  DOCUMENTS 


Symbolic!  Genera  7 2 Docamenumn,  Symbolics  Publication  Number  999079,  Symbolics, 
Inc.,  Cambridge,  Massachusetts,  1988. 


3.0  CONCEPT 

3.1  DEFINITION  OF  THE  EQUIPMENT  MODEL 

3.1.1  Purpose  and  Scope 


Ede: s the foSgbow  e^ipment  4*ed  ^support  the  Phase  IV  demonstranons. 


3.2  USER  DEFINITION 

Peoole  directly  use  the  Equipment  Model  only  for  defining  equipment  for  a simulatmm 

perform  some  specified  function,  request  components  by  name,  perform  specified 
functions,  or  tell  components  to  advance  one  time  tick. 


3.3  CAPABILITIES  AND  CHARACTERISTICS 
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^hvc^y'fn^6/11  ^odel  uses  the  P]?ysicai  and  functional  component  types  to  separate  the 
p ys  cal  and  funcdonal  aspects  of  a component.  Every  component  in  a piece  of  eaumment 
is  represented  by  both  a physical  component  type  and  a functional  component  type  The 
c°mP?nent  type,  contains  information  about  the  object's  physicd  characteristics, 
e ample,  if  a switch  is  being  modeled  then  its  physical  model  would  specify  whether  it 

f r a PUSh  bUtt°n  and’ if  il  was  3 toggIe  switch> how  much  force  would 
inf=lt^iPlfroT°nC  SJate  t0  aether.  The  functional  component  type  contains 
rmation  about  the  object  s functionality.  In  the  previous  example  the  switch's 
unctional  model  would  specify  that  the  switch  was  used  for  turning  on  and  off  a radio. 

Xf  Ut“g  3 STl3tio"' the  user  <which  is  normally  the  Symbolic  Operator  Model) 
ypically  accesses  only  the  functional  components  and  accesses  them  not  by  name  but  bv 

5 Spe^aU/' 3 user  ,wi11  *ypically issue  a command  to  the  ^uipmenffiel 
T Execute  the  functional  component  that  performs  the  following  function  " The 

functional  component  will  determine  what  physical  actions  are  required  to  execute  the 
function  and  tell  the  associated  physical  component  to  perform  those  actions. 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 

6 p£  “f  fpSS 

Mwdel'to'execute  tem5™1”'''  °Pen“°r  Model  func,io,,s  *“> K]ls  1*  Equipment 
A simple  component  definition  in  the  Longbow  Equipment  component  is 


0) 


(2) 

(3) 

(4) 


(5) 

(5) 

(5) 


USing  006  'SCapesi''<1  0 (NIL  0)  (NIL 

(2  0 (NIL  0)  (NIL  NIL  NIL)  "CPTFONT") 

, 0 
(normally-off-momentary-switch) 
rreferenced-equipment  (func-cpu) 

:specializations 

(:self-specializations 

((switch-state-output  (rts-switch-state  func-cpu)) 

(activate-switch  (activate-rts  ufd)) 

(deactivate-switch  (deactivate-rts  ufd))))) 


(The  numbered  lines  in  this  definition  are  described  below.) 

1)  The  name  of  the  component.  This  component's  name  is  RTS-SWITCH. 

2)  Internal  state  variables  of  the  component.  RTS-SWITCH  has  none  of  its  own 

but  may  inherit  some.  See  (3)  below.  ’ 

3)  More  general  components  on  which  this  one  is  built.  RTS-SWITCH  deDenrU  nn 

the  component  NORMALLY-OFF-MOMENTARY-SWITCH.  dePends  on 

4)  of  other  components  in  the  equipment  model.  These  either  supply 

inputs  to  or  accept  outputs  from  this  component.  RTS-SWITCH  refers  to  the 
component  named  FUNC-CPU.  rerers  t0  tne 
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5)  What  the  inputs,  outputs,  and  supported  functions  (in  the  more  general 
components  named  in  (3)  above)  really  refer  to.  In  this  case,  the  generic 
NORMALLY-OFF-MOMENTARY-SWITCH  component  has  an  output  called 
SWITCH-STATE-OUTPUT.  For  an  RTS-S WITCH,  this  really  refers  to  the 
RTS-SWITCH-STATE  of  the  FUNC-CPU  object.  Also,  thei [unctions ^Wtte 
Switch"  and  "Deactivate  Switch"  supported  by  the  generic  NORMALXY-OFF 
MOMENTARY-SWITCH  component  become  the  more  specific  Activate  tfte 
RTS  Switch  of  the  UFD"  and  "Deactivate  the  RTS  Switch  of  the  UFD  functions. 


This  definition  creates  the  RTS-SWITCH  component  type.  To  actually  create  an 
RTS -SWITCH  component  object,  the  user  executes: 


(make-instance  'RTS-SWITCH  . . 

cinitializations  for  RTS-SWITCH  instance  vanables>) 


Nnfp  that  the  new  RTS-SWITCH  object  automatically  gets  inserted  into  the  equipment 

S cfZfs "he vSe of to speciaA'ariable *EQW^NT-MODeA  I. MriWg 

name,  RTS-SWITCH,  and  by  function,  (ACTTV ATE-RTS  UFD)  and  (DEACTIVATE  R 
UFD). 

At  runtime  the  Symbolic  Operator  Model  may  find  that  it  needs  to  activate  the  UFD's  RTS 
switch.  It  retrieves  all  components  of  the  equipment  model  that  support  that  function  (there 
is  only  one  in  this  case),  selects  the  one  best  suited  to  its  needs,  and  executes  it. 


4.0  REQUIREMENTS 


4.1  REQUIREMENTS  SPECIFICATION 


There  are  five  major  requirements  levied  on  the  Equipment  Model: 


1)  to  describe  the  functional  and  physical  characteristics  of  the  equipment  used  in  the 
MIDAS  simulation  with  enough  detail  for  the  Symbolic  Operator  Model  to  be  able 
to  interact  with  a simulation's  equipment. 


2)  to  allow  a simulation's  equipment  to  be  built  from  libraries  of  existing  generic 
components,  rather  than  having  to  be  built  from  scratch  each  time  they  are 
needed. 


3)  to  allow  the  functional  and  physical  characteristics  of  the  equipment  to  be 
specified  independently,  so  that  physical  components  can  be  replaced  by  other 
physical  components  (with  the  same  functionality)  without  requmng 
modifications  to  the  rest  of  a simulation. 

4)  to  allow  the  Symbolic  Operator  Model  to  dynamically  choose  from  the  applicable 
functional  components  of  a simulation  which  should  be  executed. 

5)  to  represent  the  Longbow  equipment  needed  to  support  the  Phase  IV  demo. 


4.2  EXTERNAL  INTERFACE  REQUIREMENTS 


The  external  interface  to  people  is  only  required  to  support  equipment  component  definition, 
not  execution.  The  users  shall  be  able  to  define  equipment  components  by  typing  the 
appropriate  definitions  into  text  files  and  executing  them. 
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The  external  interface  to  the  Symbolic  Operator  Model  shall  allow  it  to  retrieve  and  execute 
components  by  both  name  and  functionality.  The  Symbolic  Operator  Model  can  do  this  by 
calling  the  appropriate  methods  of  the  Equipment  Database. 

4.3  IMPLEMENTATION  CONSTRAINTS 

The  Equipment  Model  requires  a Symbolics  computer  with  Genera  7.2  system  software. 

5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

5.1.1  Architectural  Design  Description 

The  Equipment  Model  comes  in  three  parts:  one  is  a way  of  defining  new  equipment 
components  and  specifying  their  physical  and  functional  properties  (the  Equipment 
Definition  module);  another  part  is  a database  indexing  the  components  by  both  their  names 
and  their  functional  properties  (the  Equipment  Database  module);  the  final  part  is  the  actual 
Longbow  equipment  model  (the  Longbow  Equipment  module).  The  Equipment  Definition 
module  allows  for  the  definition  of  generic  components  that  can  be  easily  specialized  into  the 
particular  components  needed  in  a simulation.  The  Equipment  Database  module  is 
implemented  as  an  object  with  two  components:  the  Equipment  Name  Database  and  the 
Equipment  Model  Database.  The  Equipment  Name  Database  stores  the  equipment 
components  indexed  by  their  names,  while  the  Equipment  Model  Database  stores  them 
indexed  by  their  function.  These  two  components  are  described  in  more  detail  below.  The 
Longbow  Equipment  module  contains  the  definitions  of  the  Longbow  helicopter's 
equipment  components. 

5.2  DETAILED  DESIGN 

5.2.1  The  Equipment  Model 

This  section  describes  each  of  the  components  of  the  Equipment  Model. 

5.2. 1.1  Equipment  Definition  module 

The  Equipment  Definition  module  contains  the  fundamental  object  types  which  underlie  all 
equipment  components  in  a model,  as  well  as  the  functions  and  macros  needed  to  define  the 
pieces  of  equipment  in  a given  model.  There  are  two  object  types  that  are  used  by  all 
equipment  components:  a physical  component  type  (called  Physical-Component)  and  a 
functional  component  type  (called  Equipment-Component).  Associated  with  these  are  two 
macros  used  to  define  components:  one  macro  for  defining  physical  components  (Def-Phys- 
Comp-Type)  and  one  for  defining  functional  components  (Def-Equip-Comp-Type). 

5. 2. 1.1.1  Physical-Component  Type 

The  Physical-Component  type  is  a component  flavor  of  all  physical  components  in  an 
equipment  model.  As  outlined  in  section  3.3  above,  it  interacts  with  the  user  mainly 
through  the  functional  model.  Since  the  physical  components  are  generic,  there  must  be  an 
abstract  interface  between  them  and  the  functional  components.  This  interface  consists  of 
specified  sets  of  inputs  and  outputs,  the  allowed  values  for  them,  and  the  methods  for 
mapping  between  the  functional  and  physical  states.  In  addition,  the  user  needs  to  know 
what  physical  actions  (e.g.  reaches)  are  required  to  achieve  physical  states.  The  Physical- 
Component  type  therefore  supports  a mapping  from  physical  states  to  physical  actions. 
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5.2. 1.1.2  Physical-Component  Instance  Variables 

The  Physical-Component  type  contains  the  following  instance  variables: 

1)  The  associated  functional  component. 

2)  The  set  of  physical  inputs. 


3)  The  set  of  physical  outputs. 

4)  The  mapping  from  functional  inputs  and  outputs  to  physical  inputs  and  outputs. 

5.2. 1.1.3  Physical-Component  Methods 

Physical-Component  type  supports  the  following  major  methods: 

1)  Map  an  abstract  state  to  a physical  state. 

2)  Map  a physical  state  to  an  abstract  state. 

3)  Map  a physical  state  to  a physical  action. 

4)  Execute  the  current  clock  tick's  actions. 


5.2. 1.1.4  Equipment-Component  Type 

The  Equipment-Component  type  is  a component  flavor  of  all  functional  c^P°"ents  It 
supplies  the  main  execution  imerfaee  between  a user  and a* eqmpmen  model.  As  such  ,t 
principle  interaction  with  a user  is  the  execution  of  its  individual  funcuons. 

5. 2. 1.1. 5 Equipment-Component  Instance  Variables 

The  Equipment-Component  type  contains  the  following  instance  variables. 


1)  The  component's  name. 

2)  The  associated  physical  component. 

3)  The  functions  that  the  component  supports. 

4)  The  set  of  functional  inputs. 

5)  The  set  of  functional  outputs. 

6)  The  mappings  from  functional  inputs  and  outputs  to  abstract  inputs  and  outputs. 

5. 2. 1.1.6  Equipment-Component  Methods 

Equipment-Component  type  supports  the  following  major  methods: 

1)  Map  a functional  input  to  an  abstract  input. 

2)  Map  an  abstract  output  to  a functional  output. 
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3)  Execute  a specified  function. 

4)  Execute  the  current  clock  tick's  actions. 

5.2.1.2  Equipment  Database  module 


Pr  1 D t ,b  m«lule  contains  the  object  type  that  is  used  to  build  the  equipment 
databases  for  a simulation.  It  also  contains  the  functions  and  methods  needed  to  update 
query,  and  execute  the  contents  of  the  databases.  ^ ’ 

5.2. 1.2.1  Equipment  Database  Type 

Each  Equipment  Database  type  is  used  to  represent  equipment  models.  It  is  possible  to  have 
™etfuipmement  mOC^e  S in  a s n®le  simulation,  with  each  representing  a single  major  piece 


5.2. 1.2.1. 1 Equipment  Database  Instance  Variables 

The  Equipment  Database  type  contains  the  following  instance  variables: 

1)  The  Equipment  Name  Database,  which  contains  a database  that  indexes  the 
equipment  components  by  their  names. 

2)  The  Equipment  Model  Database,  which  contains  a database  that  indexes  the 
equipment  components  by  their  functions. 

5.2.1. 2. 1.2  Equipment  Database  Methods 

Equipment  Database  type  supports  the  following  major  methods: 

1)  Graph  the  contents  of  the  Equipment  Database. 

2)  Retrieve  all  equipment  components  with  a function  that  matches  a pattern. 

3)  Retrieve  the  equipment  component  with  a given  name. 

4)  Create  an  equipment  component  of  a given  type  and  add  it  to  the  Equipment 


5)  Execute  the  function  of  the  first  equipment  component  that  matches  a given 
pattern.  ® 


5.2.1.2.2  Equipment  Name  Database  Type 


The  Equipment  Name  Database  indexes  the  equipment  components  by  their  names.  It  is 
used  when  the  Symbolic  Operator  Model  needs  access  to  specific  pieces  of  equipment  It  is 

implemented  as  a standard  system  hash  table.  4 F 


5.2.1.2.2.1  Equipment  Name  Database  Instance  Variables 

The  Equipment  Name  Database  type  contains  no  instance  variables  other  than  those  of  a 
standard  system  hash  table. 
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5.2.1.2.2.2  Equipment  Name  Database  Methods 

T„e  Equipment  Name  Database  we  supports  no  methods  other  than  those  of  the  standard 
system  hash  tables. 

5.2. 1.2.3  Equipment  Model  Database  Type 

The  Equipment  Model  Database  " 

SSSSffJsSMSS&ss" 

5.2.1. 2.3.1  Database  Node  Type 

A Database  Node  is  a single  node  in  the  key  stored 

Equipment  Model  Database  It  contains  key, data  an^wOTkmio  ^ 

in  a single  Database  Node  object  represents  ( ^3^  Ust  of  keys  of  all 

SSte ^.Sr^dTof  the  Equipment  Mode.  Database  and  Ore  node  in  quesuon. 

5. 2. 1.2. 3. 1.1  Database  Node  Instance  Variables 

The  Database  Node  type  contains  the  following  instance  variables: 

1)  The  terminal  component  of  the  key  for  the  data  stored  at  this  node. 

2)  The  data  corresponding  to  the  pattern  fomted  by  the  list  of  all  keys  between  this 
node  and  the  Equipment  Model  Database  s root  node. 

3)  The  children  of  this  node. 

5.2. 1.2.3. 1.2  Database  Node  Methods 

The  Database  Node  type  supports  the  following  major  methods: 

D Match  a pattern  with  the  key  of  a single  database  node. 


2)  Insert  data  corresponding  to  a pattern. 

3)  Insert  data  corresponding  to  a linearized  pattern. 

4)  Push  data  onto  a list  of  existing  data  corresponding  to  a pattern. 

5)  Push  data  onto  a list  of  existing  data  corresponding  to  a linearized  pattern. 

6)  Retrieve  the  data  corresponding  to  a pattern. 

7)  Search  a network  of  database  nodes  starting  from  a root  to  And  the  database  node 
corresponding  to  a pattern. 

8)  Apply  a function  to  every  node  in  a database. 

9)  Apply  a function  to  every  node  in  a database  that  satisfies  a predicate  and  collect 
the  results  of  the  function  application. 
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10)  Draw  a graph  of  the  database  network. 

5.2. 1.3  Longbow  Equipment  module 


The  Longbow  Equipment  module  contains  the  definitions  of  the  Lonahnu/ 

1)  The  Central  Processor 


2)  The  Communication  Control  Panels 

3)  The  Keyboards 


4)  The  Up-Front  Displays 

5)  The  Multi-Function  Displays 

5.2. 1.3.1  The  Functional  CPU  Type 

rn2'ed- by  - -w*  » a 

modeled  by  the  Function  CPU  type  Most  of  the  nfh°W  ^.ulPment  m°dule  this  computer  is 


5. 2. 1.3.1. 1 Functional  CPU  Instance  Variables 

The  Functional  CPU  type  has  the  following  instance  variables: 


1)  The  current  state  of  the  pilot’s  RTS  switch 

2)  The  previous  state  of  the  pilot's  RTS  switch 

3)  The  current  state  of  the  pilot’s  LAST  switch 

4)  The  previous  state  of  the  pilot's  LAST  switch 


5)  The  currently  pressed  key  on  the  pilot’s  keyboard 

6)  The  previous  state  of  the  pilot's  keyboard 

7)  The  contents  of  the  pilot's  keyboard  buffer 

8)  Whether  the  pilot's  keyboard  buffer  is  complete 

9)  The  current  state  of  the  copilot-gunner's  RTS  switch 

10)  The  previous  state  of  the  copilot-gunner's  RTS  switch 

1 1)  The  current  state  of  the  copilot-gunner's  LAST  switch 

12)  The  previous  state  of  the  copilot-gunner's  LAST  switch 
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13)  The  currently  pressed  key  on  the  copilot- gunner's  keyboard 

14)  The  previous  state  of  the  copilot-gunner  s keyboard 

15)  The  contents  of  the  copilot-gunner's  keyboard  buffer 

16)  Whether  the  copilot-gunner's  keyboard  buffer  is  complete 

17)  The  current  state  of  the  copilot-gunner's  mfd-button 

18)  The  previous  state  of  the  copilot-gunner's  mfd-button-last 

19)  The  current  page  on  the  copilot-gunner  s mfd 

20)  The  current  vhf  frequency. 

21)  The  previous  vhf  frequency. 

22)  The  current  uhf  frequency. 

23)  The  previous  uhf  frequency. 

24)  The  current  fml  frequency. 

25)  The  previous  fml  frequency. 

26)  The  current  fm2  frequency. 

27)  The  previous  fm2  frequency. 

28)  The  current  vhf  call  sign. 

29)  The  previous  vhf  call  sign. 

30)  The  current  uhf  call  sign. 

31)  The  previous  uhf  call  sign. 

32)  The  current  fml  call  sign. 

33)  The  previous  fml  call  sign. 

34)  The  current  fm2  call  sign. 

35)  The  previous  fm2  call  sign. 

36)  The  current  transmitter  selected  by  the  pilot. 

37)  Whether  the  pilot  has  selected  the  vhf  receiver. 

38)  Whether  the  pilot  has  selected  the  uhf  receiver. 

39)  Whether  the  pilot  has  selected  the  fml  receiver. 


Page  D-9 


40)  Whether  the  pilot  has  selected  the  fm2  receiver. 

41)  The  cunem  transmitter  selected  by  the  copilot-gunner. 

42)  Whether  the  copilot-gunner  has  selected  the  vhf  receiver. 

43)  Whether  the  copilot-gunner  has  selected  the  uhf  receiver. 

44)  Whether  the  copilot-gunner  has  selected  the  fml  receiver. 

45)  Whether  the  copilot-gunner  has  selected  the  fm2  receiver. 
5. 2, 1.3.1. 2 Functional  CPU  Methods 


The  Functional  CPU  type  has  the  following  methods: 

1)  Execute  the  current  time  step’s  activities.  This  consists  of: 

- Processing  new  inputs  from  the  pilot's  keyboard. 

- Processing  new  inputs  from  the  copilot-gunner's  keyboard. 

- Processing  new  inputs  from  the  pilot's  RTS  button. 

- Processing  new  inputs  from  the  copilot-gunner's  RTS  button. 

- Processing  new  inputs  from  the  pilot's  LAST  button. 

- Processing  new  inputs  from  the  copilot-gunner's  LAST  button. 
5.2. 1.3.3  The  CCP  Type 


5.2. 1.3.3. 1 CCP  Instance  Variables 


The  CCP  type  has  the  following  instance  variables: 


1)  The  VHF  receiver  control. 


2)  The  UHF  receiver  control. 


3)  The  FM1  receiver  control. 


4)  The  FM2  receiver  control. 

5. 2. 1.3. 3. 2 CCP  Methods 

The  CCP  type  does  not  support  any  methods  beyond  those  of  the  Equipment  Component 
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5.2.1.3.4  The  CCP  VHF  Receiver  Control  Type 

The  Communication  Control  Panel's  VHF  Receiver  Control  setocte  *cW  radio Tor 
listening  and  controls  its  volume.  Only  the  selection  function  is  implemented  in  the 
Longbow  Equipment  module. 

5 2.1.3.4.1  CCP  VHF  Receiver  Control  Instance  Variables 
The  CCP  VHF  Receiver  Control  type  has  the  following  instance  variables. 


1)  The  state  of  the  control. 

2)  The  Functional  CPU  object. 

5.2. 1.3.4. 2 CCP  VHF  Receiver  Control  Methods 
The  CCP  VHF  Receiver  Control  type  has  the  following  methods: 


1)  Select  the  VHF  radio  for  listening. 

2)  Unselect  the  VHF  radio  for  listening. 

5.2. 1.3.5  The  CCP  UHF  Receiver  Control  Type 

The  Communication  Control  Panel's  UHF  Receiver  Control  selects  the  UHF  radio  for 
listening  and  controls  its  volume.  Only  the  selection  funcnon  is  implemented  in  the 
Longbow  Equipment  CSC. 

5 2 1.3.5. 1 CCP  UHF  Receiver  Control  Instance  Variables 
The  CCP  UHF  Receiver  Control  type  has  the  following  instance  variables: 


1)  The  state  of  the  control. 

2)  The  Functional  CPU  object. 

5.2.1. 3.5.2  CCP  UHF  Receiver  Control  Methods 

The  CCP  UHF  Receiver  Control  type  has  the  following  methods: 

1)  Select  the  UFIF  radio  for  listening. 

2)  Unselect  the  UHF  radio  for  listening. 

5.2. 1.3.6  The  CCP  FM2  Receiver  Control  Type 


The  Communication  Control  Panel's  FM2  Receiver  Control  selects  the  FM2 .radio  for 
listening  and  controls  its  volume.  Only  the  selection  function  is  implemented  in  the 
Longbow  Equipment  module. 


5.2.1.3.6.1  CCP  FM2  Receiver  Control  Instance  Variables 


The  CCP  FM2  Receiver  Control  type  has  the  following  instance  variables: 
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1)  The  state  of  the  control. 

2)  The  Functional  CPU  object. 

5.2. 1.3. 6. 2  CCP  FM2  Receiver  Control  Methods 
The  CCP  FM2  Receiver  Control  type  has  the  following  methods: 

1)  Select  the  FM2  radio  for  listening. 

2)  Unselect  the  FM2  radio  for  listening. 

5.2.1. 3. 7 The  CCP  FM2  Receiver  Control  Type 

The  Communication  Control  Panel’s  FM2  Receiver  Control  selects  the  FM2  radio  for 
listening  and  controls  its  volume.  Only  the  selection  function  is  implemented  in  the 
Longbow  Equipment  module. 

5.2. 1.3.7. 1 CCP  FM2  Receiver  Control  Instance  Variables 
The  CCP  FM2  Receiver  Control  type  has  the  following  instance  variables: 

1)  The  state  of  the  control. 

2)  The  Functional  CPU  object. 

5.2. 1.3.7. 2 CCP  FM2  Receiver  Control  Methods 
The  CCP  FM2  Receiver  Control  type  hast!.  Ldlov.i.ig  methods: 

1 ) Select  the  FM2  radio  for  listening 

2)  Unselect  the  FM2  radio  for  listening. 

5. 2. 1.3. 8 The  Keyboard  Type 

Both  crew  stations  have  an  alphanumeric  keyboard  in  the  Longbow  cockpit.  This  keyboard 
has  50  keys,  a 44  character  buffer  with  a 22  dura;  -;-r  display,  and  a display  brightness 
control.  The  keyboard  model  has  40  keys  (A-Z,  0-9,  period,  ESC,  CLR,  and  ENTER)  and 
no  brightness  control.  The  character  buffer  and  display  are  modeled  in  the  Central 
Processor. 

5.2. 1.3.8. 1 Keyboard  Instance  Vaiia  ::s 
The  Keyboard  type  has  the  following  instance  vana tiles. 

1)  The  keys  on  the  keyboard  (each  is  a separate  instance  variable). 

5.2. 1.3. 8. 2 Keyboard  Methods 

The  Keyboard  type  does  not  support  any  metLxls ' • 1 those  of  the  Equipment 

Component  type.  ' 
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5.2. 1.3.9  The  Key  Types 

The  various  key  types  (one  for  each  of  the  keys  on  a keyboard)  support  the  entry 
of  alphanumeric  data. 

5.2.1.3.9.1  Key  Instance  Variables 

Each  Key  type  has  the  following  instance  variables: 

1)  An  instance  of  the  Central  Processor  object. 

2)  The  state  of  the  key  (ON  or  OPT). 

5.2. 1.3. 9. 2 Key  Methods 

Each  Key  type  has  the  following  methods: 

1)  Activate  (i.e.  press)  the  key. 

2)  Deactivate  (i.e.  release)  the  key. 


5.2.1.3.10  The  UFD  Type 

The  Up-Front  Display  (UFD)  in  the  Longbow  cockpit  shows  important  communications 
status  information  and  the  current  Caution/Warmng/Advisory  messages.  There  is  one  UFD 
at  each  crew  station.  The  UFD  type  in  the  Longbow  Equipment  module  only  displays  the 
statuses  of  the  VHF,  UHF,  FM 1 , and  FM2  radios  and  allows  these  to  be  changed  with  the 
RTS  and  LAST  buttons. 


5.2.1.3.10.1  UFD  Instance  Variables 
The  UFD  type  has  the  following  instance  variables: 

1)  The  RTS  switch,  which  selects  the  radio  to  be  used  as  the  crew 
member's  transmitter. 

2)  The  LAST  switch,  which  swaps  the  previous  frequency  with  the  current 
frequency  on  the  radio  being  used  as  a transmitter. 


5.2.1.3.10.2  UFD  Methods 

The  UFD  type  does  not  support  any  methods  beyond  those  of  the  Equipment  Component 

type- 


5.2.1.3.11  The  RTS  Switch  Type 

The  RTS  switch  cycles  through  the  radios  (VHF,  UHF,  FM1,  and  FM2),  selecting  each  in 
turn  as  the  crew  member's  transmitter.  The  RTS  Switch  type  implements  this  behavior  by 
updating  the  crew  member’s  currently  selected  transmitter  in  the  Functional  CPU. 

5.2.1.3.11.1  RTS  Switch  Instance  Variables 


The  RTS  Switch  type  has  the  following  instance  variables: 
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1)  The  Functional  CPU. 

2)  The  switch’s  current  state. 

5.2.1.3.11.2  RTS  Switch  Methods 

The  RTS  Switch  type  has  the  following  methods: 

1)  Activate  (i.e.  push)  the  RTS  switch. 

2)  Deactivate  (i.e.  release)  the  RTS  switch. 

5.2.1.3.12  LAST  Switch  Type 

The  LAST  switch  swaps  the  currently  selected  frequency  and  call  sign  on  the  crew 
member  s transmitter  with  the  previously  selected  frequency  and  call  sign.  The  LAST 
Switch  type  implements  this  behavior  by  updating  the  appropriate  instance  variables  in  the 
Functional  CPU. 

5.2.1.3.12.1  LAST  Switch  Instance  Variables 

The  LAST  Switch  type  has  the  following  instance  variables: 

1)  The  Functional  CPU. 

2)  The  switch's  current  state. 

5.2.1.3.12.2  LAST  Switch  Methods 

The  LAST  Switch  type  has  the  following  methods: 

1)  Activate  (i.e.  push)  the  LAST  switch. 

2)  Deactivate  (i.e.  release)  the  LAST  switch. 

5.2.1.3.13  The  MFD  Type 

pie  Multi-Function  Display  (MFD)  in  the  Longbow  Helicopter  is  a touch-sensitive  CRT 
display  that  shows  the  status  of  and  controls  nearly  every  other  system  in  the  helicopter. 
There  is  one  MFD  at  each  crew  station.  The  information  and  control  functions  are  broken 
down  into  "pages"  and  only  one  page  can  be  displayed  on  an  MFD  at  a time.  The  pages  are 
broken  down  into  five  groups:  communications,  navigation,  weapons,  aircraft,  and  fire 
control  radar. 

The  MFD  type  implements  subsets  of  the  communications  and  navigation  pages.  The  MFD 
type  contains  a finite  state  machine  (FSM)  to  control  the  complex  relations  among  the  MFD 
pages.  This  FSM  makes  it  easier  to  specify  the  legal  transitions  between  pages  and  the 
effects  of  actions  within  a page. 

5.2.1.3.13.1  MFD  Instance  Variables 

The  MFD  type  has  the  following  instance  variables: 

1)  The  current  state  of  the  MFD's  FSM. 
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2)  The  method  used  by  the  FSM's  current  state  for  processing  its 
tick-based  activities. 

3)  The  set  of  legal  FSM  states  and  their  legal  transitions  to  new 
states. 

4)  The  currently  selected  preset  frequency. 

5)  The  buffer  for  the  manually  keyed-in  frequency. 

6)  The  manually  selected  radio. 

7)  The  currently  pressed  MFD  soft  button  (or  :OFF  if  no  button  was 
pressed  during  the  last  time  step). 

8)  The  previously  pressed  MFD  soft  button  (or  :OFF  if  no  button  is 
pressed  during  the  current  time  step). 

9)  Buffer  for  holding  things  typed  for  the  MFD. 

10)  The  MFD's  preset  frequencies. 

1 1)  The  number  of  pixels  horizontally  and  vertically  in  the  MFD  screen. 

12)  The  proportion  of  the  MFD  screen  that  corresponds  to  the  diameter  of 
the  outer  range  arc  on  the  navigation  page. 

13)  The  waypoint  selected  by  the  operator. 

14)  The  transform  between  the  MFD  navigation  page  screen  coords  and  real 
world  coords. 

15)  The  translation  from  the  MFD  world  coordinate  system  to  the  screen 
coordinate  system. 

16)  The  inverse  of  the  MFD  Nav  page  map  scale  (only  the  part  of  the  scale 
between  the  MFD  world  coordinate  system  and  the  screen  coordinate 
system). 

17)  The  distance  to  the  outer  arc  on  the  MFD's  navigation  page.  This  is 
computed  whenever  the  MFD's  range  scale  or  internal  scale  is  set. 

18)  The  scale  between  the  MFD's  navigation  page  range  scale  and  the  MFD’s 
screen  coordinate  system. 

19)  The  ownship’s  present  position. 

20)  The  ownship's  present  heading. 

21)  The  list  of  all  waypoints. 

22)  The  list  of  waypoints  currently  visible  in  the  MFD. 


PageD-15 


23)  The  list  of  all  waypoints  on  the  ownship’s  route. 

24)  The  list  of  waypoints  on  the  ownship's  route  that  have  not  yet  been 
passed. 

25)  A waypoint  that  has  been  selected  to  be  added  to  or  deleted  from  the 
ownship's  route. 

26)  The  number  of  pounds  of  fuel  per  nautical  mile  being  used  at  the 
present  time. 

27)  The  number  of  minutes  of  fuel  remaining  at  the  current  bum  rate. 
5.2.1.3.13.2  MFD  Methods 

The  MFD  type  has  the  following  methods: 

1)  Execute  the  current  time  step's  activities.  This  consists  of 
executing  the  FSM's  current  processing  method. 

2)  The  processing  method  for  entering  a waypoint's  altitude. 

3)  The  processing  method  for  entering  a waypoint's  utm  coordinates. 

4)  The  processing  method  for  entering  a waypoint  description. 

5)  The  processing  method  for  entering  a waypoint’s  id  number. 

6)  Find  the  next  unused  waypoint  id  number. 

7)  The  processing  method  for  pressing  the  "add  waypoint"  soft  button. 

8)  The  processing  method  the  waypoint  subpage. 

9)  The  processing  method  for  the  route  subpage. 

10)  The  processing  method  for  the  present  position  subpage. 

11)  The  processing  method  for  the  navigation  top  page. 

12)  The  processing  method  for  when  one  of  the  radio  buttons  (VHF,  UHF, 
on  the  manual  communications  subpage  is  selected. 

13)  The  processing  method  for  when  the  ENTER  button  on  the  manual 
communications  subpage  is  selected. 

14)  The  processing  method  for  when  the  CLEAR  button  on  the  manual 
communications  subpage  is  selected. 

15)  The  processing  method  for  when  one  of  the  digit  buttons  on  the  manual 
communications  subpage  is  selected. 

16)  The  processing  method  for  the  manual  communications  subpage. 
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17)  Update  the  mfd's  graphics  transform  from  its  relevant  instance 
variables. 


18)  The  processing  method  for  when  either  the  TUNE  FM1  or  TUNE  FM2 
thr  nrfQp.tQ  communication  page  is  pressed. 


19)  The  processing  method  for  when  the  TUNE  button  on  the  presets 
communication  page  is  pressed. 


20)  The  processing  method  for  when  the  PRESET  button  on  the 
communications  top  page  is  pressed. 

21)  The  processing  method  for  the  communications  top  page. 

22)  The  initial  processing  method  for  the  mfd. 

23)  Update  the  distance  to  the  outer  arc  on  the  mfd  s display. 

24)  Determine  whether  it  is  legal  to  make  a given  transition  from  the 
current  state. 


25)  Update  the  mfd's  FSM  state  information  for  a new  state. 

26)  Convert  world  coordinates  to  screen  coordinates 

27)  Update  the  list  waypoints  on  the  current  route  that  haven't  been 
passed  yet. 

28)  Decide  whether  a waypoint  is  visible  on  the  mfd. 


29)  Update  the  list  of  visible  waypoints. 


6.0  USER'S  GUIDE 

6.1  INSTALLATION  AND  INITIALIZATION 

S logical  pathnames.  The 

El  tSXe“  «EQM-MDLfor  the  EQUIPMENT-MODEL  system  and 
LONGBOW-MDL  for  the  LONGBOW-MODEL  system. 

6 1 1 Installing  the  EQUIPMENT-MODEL  System 
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the  directory  >MIDAS>BASIOEQUIP>.  The  file  should  then  be  edited  so  that  the 
system’s  files  are  put  in  the  desired  directories.  6 

6.1.2  Installing  the  LONGBOW-MODEL  System 

The  logical  host .for  the -LONGBOW-MODEL  system  is  LONGBOW-MDL.  The  host  is 

cvctmran^  flC  ^ONGBOW'^ODELS  Y STEM,  which  must  be  loaded  before  this 
system  can  be  used.  It  is  currently  in  the  directory  B:>MIDAS>EQUIP>LONGBOW> 

LONGBOW  Monpt  ^ defuutlon  of  the  LONGBOW-MODEL  system.  The  files  for  the 

MDLdTD^BOW^**- ""  SyStCm  St0red  Under  the  directory  tree  "LONGBOW- 

When  lnstalhng  the  LONGBOW-MODEL  System,  the  LONGBOW-MODEL  SYSTEM  file 

system  s files  are  put  in  the  desired  directories. 

6.1.3  Initializing  the  EQUIPMENT-MODEL  System 

System  is  compiled  with  the  standard  system  compilation 
commands.  For  example  suppose  the  system  is  loaded  on  Barracuda.  At  a "Command  " 

MODEL  system: fl  C EQUIPMENT'MODEL  SYSTEM  and  then  compile  the  EQUIPMENT- 

MODEL SYSTEM1 L°ad  FUe  B:>MIDAS>BASIC>EQlJIP>EQUIPMENT- 

Command:  Compile  System  EQUIPMENT  MODEL 
If  the  system  is  already  compiled,  then  you  can  load  it  as  in: 

MODEL  SYSTEM ^ B:>MIDAS>BASIC>EQUIP>EQUIPMENT- 

Command:  Load  System  EQUIPMENT  MODEL  .-Version  Newest 
The  keyword  argument  ":Version  Newest"  is  required. 

6.1.4  Initializing  the  LONGBOW-MODEL  System 

^mm?^?BpW"M°DiEL  System  is  comPiled  with  the  standard  system  compilation 
commands.  For  example,  suppose  the  system  is  loaded  on  Barracuda  At  a "CnmmanH  " 

MODELsystem:  ^OW-MOoksYSTEM  a*d  '££&  & LOMBCW. 

MODEUSYSTCM^)ad  ^ B:>MIDAS>E0(JIP>I.ONGBOW>LONGBOW- 

Command:  Compile  System  LONGBOW  MODEL 
If  the  system  is  already  compiled,  then  you  can  load  it  as  in: 

MODEL'S  YSTCM  ^°ad  Rle  B:>MIDAS>EQUIP>LONGBOW>LONGBOW- 
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Command:  Load  System  LONGBOW  MODEL  rVersion  Newest 
The  keyword  argument  ":Version  Newest"  is  required. 

6.2  STARTUP  AND  TERMINATION 

The  Equipment  Model  does  not  require  any  startup  or  termination  beyond  that  described 
section  6.1  above. 
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\jtAM  m ACH1NF  INTEGRATION  DESIGN  & ANALYSIS  SYSTEM 
(MIDAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 
' PHASE  IV: 

VISUAL  EDITOR  AND  SIMULATION  TOOL 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 


SttSSS 

Software  Component  ( CSC ),  units,  or  functions  contained  within  this  module. 


1.2  SCOPE  OF  DOCUMENT 

This  document  describes  the  framework,  functions  and  use  of  the  VEST  software 
, Hurinfj  Phase  IV  of  the  A^I  program.  This  document  assumes  that  the  reade 

with  MultiGen®  CAD. 

1.3  PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 

VEST  is  developed  on  top  of  commercial  CAD  package  MultiGen.  The  basic  orations 
and  programmingtechniques  of  MultiGen  are  well  documented  in the  Software  Systems 
MulliGeTZuals.  The  purpose  of  this  document  is  to  show  the  results  of  VEST 

accomplished  during  Phase  IV  of  the  A3I  program  to 

module/component  layout,  animation  interface,  and  internal  structures  as  the  need 

modify  the  source  code  may  arise. 

2.0  RELATED  DOCUMENTATION 


2.1  APPLICABLE  DOCUMENTS 

The  following  documents  are  referenced  herein  and  are  directly  applicable  to  this  volume: 


Software  Systems  MultiGen®  - Modeler's  Guide,  Interface  Manager,  MultiGen  Kernel, 
Writing  MultiGen  DBL,  Release  3.0,  Software  Systems,  San  Jose,  September  1988. 


Data  Format  Specification  for  Software  Systems  Flight  Data  Bases,  Format  Release  4,  July 

28,  1988. 


Software  Systems  MultiGen®,  DMA  Terrain  Conversion  Option  User's  Guide,  July 

1988. 
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A3I  Phase  III  Views  CSCI,  June  1990. 

A3I  Phase  III  Cockpit  Design  Editor  CSCI,  June  1990. 
A3I  Phase  III  JACK  CSCI,  June  1990. 

APACHE  longbow  documentation. 

2.2  INFORMATION  DOCUMENTS 


The  following  documents  amplify  or  clarify  the  infonnation  presented  in  this  volume: 

E?g"e  woJd'ciiffeN.J^  wf  M RiIChie' m r ’'^ramming  Language.  Prentice-Hall. 


w«iFo1' 7 A sV/n  Da,m'  Ftmdanvintals  of  Interactive  Computer  Graphics,  Addison- 

Wesley,  Reading,  Massachusetts,  1982. 


Silicon  Graphics  Inc.,  IRIS  User's  Guide, 
California,  1986. 


Volume 


and  II,  Version  3.0,  Mountain  View, 


Silicon  Graphics  Inc.  IRIS  Programmer's  Manual,  Volume  IB,  System  Calls  and 
Subroutines,  Version  2.1,  Mountain  View  California,  1986. 

3.0  CONCEPT 

3.1  DEFINITION  OF  SOFTWARE 

3.1.1  Purpose  and  Scope 

The  purpose  ol  VEST  is  to  develop  a set  o'  visual  tools  to  support  the  A3I  simulation 
world.  It  provides  an  environment  for  flight  simulation,  instrument  animation,  and  Jack 
movement.  To  rapidly  create  3D  graphical  objects  for  simulation,  MultiGen®  modeling 
system  is  adopted  for  its  ease  of  use  and  VEST  is  implemented  as  a subsystem  of  it  The 
software  described  here  contains  MultiGen®  modules,  DMA,  VIEWS,  CDE  MFD  and 

CDF  el  f c rtasj?es  Phaf  TV  eRbit  on  VIEWS,  MFD  and  Jack-Interface 

pan  of  SS*  package’'”5  ” 'locumEnted  in  '"“d  is  now 

3.1.2  Goals  and  Objectives 

The  goals  of  VEST  are: 


(0  to  create  animation  databases  for  Multi  bum  i ion  Displays  (MFD)  and  traditional 
gauges.  The  values  of  their  dynamic  parameieis  can  be  assigned  so  that  their 
behaviors  can  be  monitored  during  simulation 


® .to  ^isPlay  simulation  in  different  viewing  perspectives  with  the 
of  the  Mission  Simulation. 


progression 


(iii)  to  provide  an  interface  with  Jac;  soiivait-  \n  animate  aviator's  reaches. 


Pare  F-? 


(iv)  to  provide  a convenient  interface  for  integrating  the  graphics  system  with  the 
simulation  network. 

3.1.3  Description 

VEST  attempts  to  provide  a user  tools  to  create  3D  graphical  objects  and  to  display  them 
based  on  data  received  from  the  simulation  models.  It  runs  on  SGI  IRIS  workstations  at  a 
cost  well  below  those  of  visual  simulation  systems  and  incorporates  interacuve  graphic 
tools  for  constructing  and  displaying  ihe  Mission  Simulation  at  an  acceptable  speed.  In  t ie 
animation  mode,  it  communicates,  through  the  network,  with  the  Executive  which  contains 
a datapool  available  for  all  processes. 

3.2  USER  DEFINITION 

VEST  is  developed  for  cockpit  designers.  However,  any  users  with  some  experience  with 
SGI  IRIS  workstations  can  use  VEST  to  create  and  edit  3D  graphical  objects.  The  use  of 
the  DMA  component  needs  knowledge  of  DMA  tape,  UNIX  and  MultiGen®.  To 
effectively  use  CDE  and  MFD  modules , users  need  to  know  instrument  characteristics, 
network  interface  with  the  communication  manager,  and  some  data  structure  of  the 
datapool.  Jack  interface  requires  experience  with  Jack  and  MultiGen®.  The  use  of 
VIEWS  is  based  on  the  organization  of  the  MultiGen®  database  tree-structure  and  the 
orientation  of  graphical  objects. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

The  following  are  VEST  modules  with  their  capabilities: 

1.  MultiGen®  modeling  system:  a CAD  package  developed  by  Software  Systems 
to  create  and  edit  3D  geometrical  objects. 

2.  DMA:  a subsystem  to  generate  terrain  surfaces  from  any  standard  Level  1 DMA 
DTED  tape.  It  was  completed  in  Phase  IE. 

3.  CDE:  a subsystem  to  animate  traditional  instruments.  Most  of  its  components 
were  completed  in  Phase  III.  Extensions  are  made  to  display  character  strings,  to 
correct  the  z-buffer  problem,  and  to  link  instruments  to  dynamic  data  pool. 

4.  MFD:  a subsystem  to  create  MFD  pages  and  simulate  them.  Most  components 
are  based  on  APACHE  longbow  configuration. 

5.  Jack  Interface:  a component  to  display  and  animate  Jack  images/objects  in 
MultiGen®  window  environment. 

6.  VIEWS:  a module  to  change  camera  views  and  control  data  flow  between 
VEST  and  network  simulation  manager. 

VEST  operates  within  MultiGen®  and  thus  uses  MultiGen®  windows  for  display  and  user 
interface.  To  run  dynamic  animation,  communication  network  must  be  established  between 
VEST  and  Simulation  Executive.  However,  with  some  modification  to  the  datapool 
interface,  it  can  be  used  to  animate  any  graphic  images.  In  that  regard,  VEST  can  also  run 
standalone  animations. 


Page  E-3 


3.4  SAMPLE  OPERATIONAL  SCENARIOS 

VEST  is  a Mac-like  software  on  top  of  MultiGen®  user  interface  and  is  thus  user  friendly 
and  mouse-dnven  . Using  the  menu  bar  and  icons,  the  user  specifies  commands  for 
VEST.  Through  dialog  boxes,  the  user  is  prompted  for  inputs.  The  user  first  creates 
graphical  objects  in  an  organized  tree  structure  with  the  animation  in  mind.  Then,  the 
designer  uses  CDE  or  MFD  tools  to  link  the  graphical  objects  to  datapool  variables  and 
starts  animation  through  VIEWS  functions. 

4.0  REQUIREMENTS 

4.1  REQUIREMENTS  APPROACH  AND  TRADEOFFS 

VEST  is  developed  to  provide  an  intuitive  look  at  the  progression  of  the  Mission 
Simulation  for  cockpit  designers  and  mission  analysts.  As  a result,  its  requirements  come 
from  the  needs  of  displaying  the  A3I  simulation  at  an  acceptable  speed.  As  a prototyping 
tool,  MultiGen®  is  adopted  as  the  underlying  CAD  software  for  rapidly  creating  3D 
graphical  objects.  The  tradeoff  is  that  a license  fee  is  needed  for  use  of  MultiGen® 
software.  To  model  APACHE  longbow  MFDs  which  are  not  in  production  yet,  updated 
documention  is  needed  from  McDonnell  Douglas  Helicopter  Company. 

4.2  EXTERNAL  INTERFACE  REQUIREMENTS 

VEST  is  developed  to  run  on  SGI  IRIS  graphics  workstations  and  Mission  Simulation  to 
run  on  Symbolics  machines.  To  support  the  A3I  integration  world,  VEST  is  required  to 
communicate  with  the  Simulation  Executive.  This  network  interface  is  derived  from  the 

message  formats  and  commands  defined  by  the  Executive. 

VEST  needs  to  be  linked  to  part  of  Jack  software.  An  interface  is  required  to  resolve  the 
difference  in  orientation  and  scale.  A solution  to  this  is  to  keep  Jack  drawing  routines  from 
MultiGen®  and  define  an  interface  matrix.  Before  any  Jack  drawing  or  reach,  the 
MultiGen®  world  is  modified  by  this  interface  matrix  to  let  Jack  perform  it  in  its  own 
environment.  Since  MultiGen®  has  its  own  color  setup,  the  attributes  of  Jack  sites  need  to 
be  disabled. 

For  better  performance,  VEST  is  implemented  on  IRIS  4D/GT  workstations. 

4.3  REQUIREMENTS  SPECIFICATION 
4.3.1  Process  and  Data  Requirements 

VEST  is  required  to  be  a process  driven  by  the  mouse  and  network  communication.  It  runs 
in  a loop  waiting  for  mouse  events  or  network  messages.  When  a mouse  is  pressed,  VEST 
operates  within  MultiGen®  and  takes  user’s  commands  through  the  pull-down  menu  or 
icons.  It  opens  dialog  boxes  to  prompt  the  user  for  more  inputs  when  necessary.  When  a 
network  message  arrives,  it  follows  the  Executive's  protocol  and  performs  necessary 
actions.  For  each  tick,  VEST  informs  the  Executive  of  Jack  status. 
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MultiGen®  provides  a set  of  functions  to  a user  to  generate  3D  graphical  objects.  As  a 
subsystem,  VEST  provides  the  following  major  functions,  through  the  pull-down  menu 
meet  Phase  IV  requirements: 


,to 


CDE* 

Gauge  Library:  to  create  3D  graphical  objects  of  gauges  from  library  tools. 
Animation  Linking:  to  define  animation  methods  and  link  animation  to  the 
datapool  variables. 

Color  Table:  to  animate  changes  in  color  intensity  of  gauges. 


VIEWS:  , , „ . 

Communication:  to  open  a network  socket  to  the  Executive. 
Camera  Views:  to  attach  cameras  to  moving  objects. 

Setup  Script:  to  set  up  camera  views  from  a script  file. 


MFD: 


JACK: 


Editing:  to  open  the  library  tools  to  build  MFD  pages. 

Structure:  to  modify  the  MFD  tree  and  pages. 

Animation:  to  page  through  the  MFD  tree. 

Interface  : to  interactively  define  VEST- Jack  interface  matrix. 
Reach:  to  send  a command  to  Jack  to  reach  a MultiGen®  polygon. 


It  is  noted  that  DMA  and  most  of  CDE  were  completed  in  Phase  m.  VIEWS  was  also 
developed  in  Phase  III  but  is  restructured  in  Phase  IV  to  support  flexible  and  interactive 
views  including  the  attach  of  a camera  to  the  aviator's  eye. 


4.3.2  Performance  and  Quality  Engineering  Requirements 

VEST  provides  a designer  an  interactive  and  3D  visual  environment.  Its  performance  is 
limited  mainly  by  the  graphics  hardwares,  MultiGen®  Kernel  and  simulation  network. 
Because  of  the  enormously  large  database  in  the  A3l  simulation  world,  the  speed  of  IRIS 
4D/GT  is  sometimes  intolerable.  The  solution  to  this  is  to  reduce  the  size  and  details.  1 he 
MultiGen®  Kernel  converts  floating  numbers  into  integral  numbers.  During  the 
conversion,  some  degree  of  accuracy  is  lost  and  it  causes  graphics  problems  in  z-buffer  and 
shading.  The  version  of  MultiGen®  being  adopted  was  implemented  on  old  IRIS 
platforms  and  thus  limits  the  quest  for  realistic  graphical  images.  The  animation  of  VES I 
is  also  tied  to  the  performance  of  the  Simulation  models. 

VEST  is  written  following  MultiGen®  conventions  which  are  well  documented  in  Software 
Systems'  manuals.  It  is  portable  on  most  SGI  IRIS  platforms. 


4.3.3  Implementation  Constraints 

VEST  is  written  in  C and  uses  IRIS  Graphics  Library.  It  also  opens  a TCP/IP  socket  for 
network  communication.  It  is  implemented  under  the  UNDC  V operating  system  integrated 
with  Berkeley  (4.3  BSD)  extensions  and  C compiler. 
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5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

5.1.1  Design  Approach  and  Tradeoffs 

As  mentioned  before,  the  goal  of  VEST  is  to  support  A3I  simulation.  For  rapid 
development  as  a prototyping  package,  a set  of  library  tools  arc  built  to  simulate 
Multifunction  Displays  and  traditional  gauges.  The  result  is  that  VEST  is  not  as  flexible  as 
it  should  be  and  its  animation  capabilities  are  limited. 

VEST  is  developed  in  parallel  to  MultiGen®  Kernel.  Modules  are  designed  to  be 
independent  as  much  as  possible.  However,  to  avoid  high  cost  of  development,  CDE 
closely  interacts  with  MultiGen®  Kernel. 

5.1.2  Architectural  Design  Description 

VEST  is  designed  on  top  of  MultiGen®  which  is  a window-oriented  modelling  system  for 
generating  3D  graphical  objects.  The  structure  of  MultiGen®  is  well  documented  in 
Software  Systems'  manuals.  The  following  figure  illustrates  the  modules  of  VEST: 


Figure  1.  Modules  of  VEST 

Under  IRIS  MEX  window  manager,  the  User  Interface  Manager  accepts  all  the  commands 
from  the  user  and  passes  them  to  the  appropriate  subsystem.  MultiGen®  stores  graphical 
objects  in  the  Data  Base  Logic  (DBL)  format.  CDE  and  MFD  generate  hierarchical 
descriptive  database  for  the  designer  to  evaluate  cockpit  prototypes.  The  Jack- VEST 
interface  simplifies  many  Jack  features  and  only  supports  FLAT-shaded  images. 

5.1.3  External  Interface  Design 

When  VEST  is  connected  to  the  simulation  network,  it  assumes  no  knowledge  of  the 
history  and  actions  of  the  Executive.  It  is  completely  passive  and  takes  mouse  commands 
at  the  same  time. 
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5.2  DETAILED  DESIGN 

5.2.1  Detailed  Design  Approach  and  Tradeoffs 

VPCT  k designed  top-down.  Each  module  has  its  own  mouse-driven  event  handler  to 
Sk  events8passedPby  the  MultiGen®  Interface  manager.  Under  the  handler,  a se 
tools  are  developed. 

5.2.2  Detailed  Design  Description 
5. 2. 2.1  Compilation  Unit 

The  source  files  are  organized  in  the  home  directory  of  VEST  as  follows: 


MultiGen  User  Interface  Manager  (under  Im/): 
imcont.c  imde.c  imkb.c 

imtextx  imwind.c 

immemx 

immenu.c 

MultiGen  Kernel  (under  Mg/Ker): 
mgcom.c  mgcontx 

mgfile.c  mgicnstr.c 

mgidl.c  mgieditc 

mginfo.c  mgipick.c 

mgmain.c  mgmath.c 

mgdbvw.c 

mgiconstx 

mgifun.c 

mgistruc.c 

mgpage.c 

mgdeskx 

mgicre.c 

mgilib.c 

mgiutl.c 

mgstd.c 

mgdev2d.c 

mgidisp.c 

mgiman.c 

mgixform.c 

mgtab.c 

MultiGen  DBL  (under  Mg/Flt): 
fltcolor.c  fltdbl.c 

fltid.c 

fltlinkx 

fltpage.c 

DMA  (under  Mg/Dma): 

terdma.c  terfont7.c 

terpoltc 

terrainx 

CDE  (under  a3i /): 
cdecolor.c 
cdefunc.c 

cdedefaultx 

cdegauge.c 

cdedispx 

cdelinkx 

cdefile.c 

cdeneigx 

cdefmter.c 

cdepage.c 

VIEWS  (under  a3i/): 
a3istamp.c 

a3iutil.c 

animate.c 

aplselectx 

simdata.c 

MFD  (under  MFD/): 
mfdani.c 
mfddefault.c 
mfdhsd.c 
mfdmod.c 

mfdbut.c 

mfddesk.c 

mfdio.c 

mfdpage.c 

mfdcompassx 
mfddisp.c  mfdfield.c 

mfdliftx  mfdmainx 

mfdsprintf.c  mfdstruc.c 

mfdcont.c 

mfdflpath.c 

mfdmisc.c 

mfdtool.c 

mfdway.c 

JACK  Interface  (under  a3i/): 
a3ijcl.c 

JACK  (under  JACK/): 

assign. c delete.c 

msg.c  peaparse.y 

peabody/: 

dof.c 

globals.c 


draw.c 

psurfparse.y 

error.c 

jointx 


jclinit.c 

stamp.c 

expr.c 

keyword.c 


jclreach.c 


func.c 

metric.c 
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namex 

new.c 

reach.c 

reachsitesx 

scale.c 

segment.c 

psurf/: 

treex 

update.c 

value.c 

binx 

edge.c 

get.c 

vec/: 

normaLc 

utilx 

read.c 

reference.c 

chunkx 

findfile.c 

lexi.c 

matrixx 

mattorot.c 

metric.c 

quatemoinx 

string.c 

tomatrix.x 

reachsite.c 

symtab.c 

verify.c 

globals.c 

spec.c 


list.c 

pseud.c 

vector.c 


windowtype 

mfdcontrol 

showmfd 

showjack 


VEST  is  implemented  on  IRIS  workstations  and  linked  to  the  Graphics  Library. 

5.2.3  Detailed  Design  of  Compilation  Units 
5.2.3. 1 MultiGen®  User  Interface  Manager 

This  subsystem  is  a collection  of  procedures  that  support  a mouse-  and  window-oriented 
d^Vic  f CC;  c PF°v,des  the  interface  between  the  user  and  application  program  For 
Retea*  S3"  yS“m  MulIiGen®'  ProSrammer'sUuide  » Sac* 

Four  flags  are  added  to  the  window  structure  in  the  file  "imstrucs.h": 

a flag  to  indicate  whether  the  window  is  created  by  VIEWS 
a flag  to  indicate  whether  the  window  takes  user's  input 
a flag  to  indicate  whether  the  window  should  display  MFD  oases 
a flag  to  indicate  whether  the  window  should  display  Jack  figines 

5. 2.3.2  MultiGen®  Kernel 

This  subsystem  is  the  core  of  the  MultiGen®  CAD  package  and  contains  all  the  editine 

-^asisssfe«s^ can  found  ,n  softwarc  s>,sKras' 

MFD  displays  and  Jack  drawing.  This  display  unit  first  sets  up  the  camera  view  When  a 
window  ,s  marked  by  VIEWS,  its  camera  view  is  calculated  ftSm 

StTn? thtC  rrifVing^b^Ct'  77,6  drawin8  component  then  recursively  scans  all 

St  S wMbv  cS^%!Z*mSk?  by  VIEWS’ il  is  ngarded  38  a WOTld  <**«.  If  an 

din,  E£“  «<*-  » 

Ke™''  A3' »« m 

5.2,3. 3 MultiGen®  D3ts  Base  Logic 

This  subsystem  handles  MultiGen®  database.  Detailed  information  can  be  found  in 
Software  Systems  MultiGen®,  Programmer's  Guide  to  Writing  MultiGen®  DBL,  Release 
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5. 2.3.4  DMA 

“nSSem  Views  CSCI  and  also  in  Software  Systems'  MnldGen®,  DMA  Tetratn  Data 
Conversion  Option  User's  Guide. 

5.2.3. 5 CDE 

The  following  figure  shows  the  design  structure  of  CDE-user  interactions: 

1 Menu  Event 


CDE  Event  Handler 


— 2/ 

Parameter 

Linker 

3D  Gauge 
Generator 

"T~* 

Color 

Handler 

i 

Instrument 
Description 
■ Frlilox 


animation 

methods 


MultiGen 
if  Objects 


MultiGen 
^ Object 
Color 


CDE  Database 


g.  — | 

Neighborhood 
Analyzer 

Page/Field 

Editor 

l 

r 

CDE  Database 

Figure  2.  CDE  Design  Structure 

The  CDE  animation  is  called  by  the  MultiGen®  display  component,  as  showing  in  the 
following  figure. 


Figure  3.  CDE  Animation  Structure 

If  the  CDE  animation  involves  any  movement,  it  first  modifies  the  matrix  stack  of  IRIS  GL 
to  redefine  die  o^ect  position  before  drawing  it  and  then  restores  the  matrix  stack  after  the 
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ortheobjanmShed  If  1116  arumation  involves  a color  change,  it  redefines  the  RGB  color 

Sf  S?  fu£?,0ns^e*  c°™pleted  in  Phasc  m 311(1  documented  in  A3l  phase  HI 

ParametS  Vrf  rCi'  £Dw  module  has  30  Animation  Controller  in  "cdefile.c"  and 

Parameter  Linker  m cdelink.c  . New  animations  can  be  easily  added  to  these  two 

components  by  following  the  data  flow  and  MultiGen/CDE  convention.  As  required  CDE 
is  enhanced  capable  of  displaying  character  strings.  In  Phase  IV,  CDE  animation  updates 
graphical  displays  by  taking  values  from  the  datapool.  The  file  "cdedefault.c"  contaumhe 
ad  Jesses  and  types  of  datapool  variables.  When  requested,  this  unit  returns  the  value 
and/or  type  of  a datapool  variable. 

Note  that  because  of  the  enhancements  made,  the  CDE  cannot  read  old-version  databases 
The  following  secuons  discuss  the  extenstions  to  the  CDE. 

5. 2.3. 5.1  File  cdefile.c 

This  file  handles  the  CDE  animation.  Changes  are  made  to  the  following  routines. 

handle,s  *e  menu  eveni  If  the  event  is  for  VIEWS,  it  is  passed 
°^VIEWS  event  handler.  If  the  event  is  due  to  ihe  menu  function  Link  Animate,  it 

iUm  Ih^«C  valyel?f  dataP?°1  variables  by  alternately  opening  three  files  and  loading  them 
nto  the  datapool.  This  standalone  animation  continues  until  the  mouse  is  pressed. 

CDE  transformationO:  This  function  is  called  by  the  MultiGen®  display  unit.  It 
decides  which  animation  method  should  be  invoked.  If  the  graphical  object  is  for  string 
display,  the  following  steps  are  taken  to  ensure  the  z.  buffer  accuracy:  ® 

zwritemask(OxO); 

draw_polygonal_surface();  /*  within  MultiGen  */ 
CDE_show_str_disp();  /*  draw  the  string  */ 
zwritemask(Oxffffff); 
writemask(OxO); 

draw_polygonal_surface();  /*  draw  the  polygon  again  */ 
wntemask(Oxfff);  /*  depends  on  the  MultiGen  color  map  */ 

This  Junction  controls  most  of  the  CDE  animation  methods.  For  a string 
function11  perf°rms  the  necessary  format  conversion  and  then  calls  the  string  display 

rnF7?h^Cl h=UP( [):  ?is  fTCli0n  is  used  ^ Jack  to  determine  if  an  object  is  animated  by 
(‘he  control  sticks  can  be  animated  by  the  CDE  tools).  When  an  object  changes  the  * 

reach  ^ ^ S h£md  ‘S  °n  U’ the  new  position  must  **  obtained  before  Jack  does  the 
5. 2. 3. 5. 2 File  cdelink.c 

mh^hSe  Cc ' 1,tro1  • the  us%  interface  for  defining  the  linking  parameters  and  animation 
method.  Extensions  are  listed  as  below: 

Str_disp_setup():  This  function  opens  the  dialog  box  to  take  user’s  inputs  for  the 
mteractivdy  ^ dlSp  3 cbaracter  stnng-  It  allows  a user  to  change  the  font  size 
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CDE_show_str_disp():  This  function  displays  a character  string  on  the  screen. 

5. 2. 3. 5. 3 File  cdedefault.c 

This  file  defines  datapool  variables.  There  are  three  global  variables  available  for  all  VEST 
modules: 

dataDOol  a data  structure  same  as  the  Simulation  Executive. 
cdedefVals  a reference  id  array  used  by  the  user  to  access  the  datapool. 
commjnmsg  the  address  of  the  datapool  to  store  the  network  message. 

rnF  onpn  default  dbO-  This  function  takes  one  argument  as  the  filename.  It  opens 

the  variable  or  instrument. 

5. 2. 3. 6 VIEWS 

VIEWS  was  developed  in  Phase  III  and  restructured  in  Phase  IV  to  suPPOrt  Jyna™c 
camera  vTews  and  datapool  structure.  The  following  shows  the  flow  chart  of  VIEWS. 


Figure  4.  VIEWS  Design  Structure 

by  the  Executive  for  the  datapool  structure. 
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1 lead  helicopter 

2 wing  helicopter 

3 tank 

4 truck 

5 jeep 

6 launcher 


5L2SL*  d° the  a?,mttl0n  J0r  *c  world  obJects-  the  graphical  objects  for  a world  object 
should  be  organized  to  be  under  the  same  parent  node  in  the  MultiGen®  tree.  J 

VIEWS  assumes  that  the  nose  of  the  MultiGen®  helicopter  object  points  to  the  positive-x 
direction  It  also  assumes  that  the  north  of  the  terrain  is  in  the  positive-y  direction,  which  is 
the  case  if  the  terrain  is  extracted  by  the  DMA  module.  It  is  noted  that  the  orientation  of  a 
static  world  object  must  be  specified  through  the  menu  function  "Setup”  so  that  VIEWS 
knows  how  to  rotate  the  object  to  the  assumed  heading  direction. 

5.2. 3.6.1  File  animate.c 

VIEWS  opens  MultiGen  windows  but  assigns  a different  set  of  functions  for  control 

rSWSl°i-nTaCtlVely  adjust,cam(;ra  views- . The  world  objects  and  window  parameters 
can  be  modified  in  a setup  window.  For  each  tick,  the  moving  camera  view  is  re-calculated 
from  the  current  position  and  orientation  of  the  attached  world  object.  The  animation 
concept  for  world  objects  is  same  as  that  in  CDE.  In  addition  to  animation  VIEWS 

fromthe  Executive"  **  ^ t0  unfinished  reaches  when  a new  tick  is  sent 


AnamenuO:  This  function  handles  the  menu  event. 

A N A h i dew  i nd  o w ( ) . This  function  toggles  between  activating  and  deactivating  a 
window.  It  can  be  used  to  deactivate  a window  when  too  many  windows  are  opened  on 
the  screen.  Its  main  use  is  to  hide  the  database  window  of  the  MultiGen  when  VIEWS 
animation  is  in  progress  so  that  the  drawing  speed  can  be  increased.  Note  that  if  the  main 

,he  m'mory  space  “d  Mher  windows  "d  -°>  **  » 


ANA_process():  This  function  processes  the  script  file  to  set  up  VIEWS  windows. 

ANA  transformationO:  This  function  first  modifies  the  stack  matrix  for  the  world 

JwS’.dra^  thu'\reSt0reS  s.tack  matrix-  The  Portion  and  orientation  of  the 
r.f'iif1  taken  fromr-thj  da,aP°o1-  This  function  also  takes  account  of  the  scale  and  offset 
o the  object  as  specified  in  the  content  of  the  setup  window.  Note  that  due  to  the  assumed 
heading  direction,  there  is  a 90-degree  difference  in  the  yawing  angle  from  the  Aero  Model. 

ANA_pilotview():  The  function  is  called  by  the  display  unit  of  MultiGen®  to  set  up  the 
perspea,™  v,ew  of  a window.  From  .he  world  object  ,o  which  the  camera  ilaLS 
VIEWS  performs  the  following  steps  to  determine  the  viewing  matrix: 


i)  Determine  the  matrix  so  that  after  the  world  object  is  drawn,  it  remains  at 
the  same  position  and  orientation  in  the  IRIS  world.  This  is  the  inverse  of 
the  function  ANA_transformation".  The  effect  is  that  the  rest  of  MultiGen 
objects  are  drawn  in  the  local  coordinates  of  the  moving  world  object.  At 
this  step,  the  camera  is  fixed  with  the  helicopter. 
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ii)  The  matrix  is  modified  so  that  the  local  coordinates  of  the  camera  agree 
with  the  IRIS  world. 


iiil  The  above  matrix  is  further  modified  so  that  the  camera's  view  (local  * 
£jds)  points  to  the  minus-z  direction  and  the  camera  s top  (local  z axis)  to 
the  positive-y  direction  of  the  IRIS  world.  The purpose ^f  this 
ic  tr>  chnw  the  helicoDter  upward  on  the  screen. 


am  a Pthernet  readvO*  This  function  checks  if  the  content  in  the  setup  window  has 

Sid  then  lies  to  connect  VEST  to  the  Executive  through  a network  socket. 
ANA  communicateO:  This  function  is  called  by  the  function  IWM  waitforeventO 

cunent'values  in  the  datapool.  and  displays  the  tick  number  on  the  top  wtndow. 

ANA  ibwindowO:  This  function  follows  the  steps  utaa^he  MuWto  fmctio^ ^ 
DBVW  ibwindowO  to  create  a new  display  window.  However,  t ® 1 

new  window  deals  wid,  moving  camera  views.  The  icons  in  the 

I SSSSSssl 

indicates  the  rotation  in  x,  the  second  in  y,  and  the  third  in  z. 

ANA  search  up()  The  function  is  used  by  Jack  and  MFD  to  determine  if  a graphical 
objecfmoves  with  the  helicopter.  If  it  does,  its  world  position  must  be  calculated  be  o 
reach  or  drawing  is  performed. 

ANA  menu  focus  view():  This  function  calculates  the  camera  position  to  highlight 

the^ selected  MultiGe"n  polygon  in  a blown-up  view.  It  assumes  that  the  window  shows 
perspective  view. 

ANA  draw  route():  This  function  draws  the  dynamic  route  defined  and  updated  in  the 

datapool.  It  draws  a straight  line  between  two  waypoints. 

ANA  terrain  coordO:  This  function  calculates  the  relative  location  of  a MultiGen  point 

(external  to  the  moving  helicopter)  in  the  local  coordinate  system  of  the  ying  p 

ANA  get  win  fig():  This  function  returns  the  Jack  figure  to  whose  eye  the  camera  is 

attached.  It  is  used  when  the  eye-view  flag  is  set  for  a window. 
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5.2. 3.7  MFD 

The  following  figure  shows  the  design  of  MFD: 


Figure  5.  MFD  Design  Structure 


The  display  of  MFD  is  invoked  by  the  MultiGen  display  component: 

display 


Figure  6.  MFD  Display  Structure 
5.2.3.7.1  File  mfdani.c 


This  file  contains  functions  to  page  through  the  MFD  tree, 
for  a MFD  button  and  performs  the  page  switch  when  the 


It  issues  a Jack  reach 
reach  is  done. 


command 
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MFD  proc  bid_map():  This  function  processes  the  content  in  the  page-map  window. 

It  assigns  anlnitial  display  page  to  a MFD. 
procedure  for  mouse  input  is  MMFD_animate_func()  . 

MFD  setjmipagesf):  After  a reach  is  finished,  this  function  is  called  by  Jack  module 
to  assign  a page  to  a MFD. 

MFD  Mt_treenode():  This  functiOTP^onns^he^wtuch^fOT  a page-in  the^M^Juee 

$S?e$  ahiS^selLtes  the  fhild  p%cs  of  the  cunent  displaying  page  to  ftnd  one 
that  matches  the  button  id. 

[Si"e^^ 

Se  function  "ANA_animate_func()"  when  a button  is  pressed. 

5. 2.3.7. 2 File  mfdbut.c 

This  file  contains  functions  to  create  soft  buttons  (see  Section  5.2.3.7.14)  and  to  calculate 
their  3-D  locations  for  Jack  reaches. 

MFD  reach  sbut.onO:  This  function  first  calculates  the  3-D  location  of  a button  torn 
the  2-D  MFDdatabase  and  then  issues  a Jack  reach  command. 

MFD_reach_field():  This  function  determines  the  3-D  location  of  a field  in  the 
displaying  MFD. 

5. 2. 3. 7.3  File  mfdcont.c 

This  file  contains  functions  to  identify  MFD  screens  from  MultiGen  objects. 

MFD  base():  This  function  pops  up  a dialog  box  to  identify  a MultiGen  object  as  a MFD 
(through  the  picking  function)  and  its  integral  id. 

the  page  appears  on  the  MFD  screen. 

MFD  refangleO:  This  function  calculates  the  angles  that  can  be  used  to  rotate  a vector 
about  x and  y axes  to  align  it  with  the  z axis. 

MFD  adjustxO:  Given  a surface  and  the  results  for  its  normal  v^“r^tJeexfu"  J0" 
"MFDLrefEmgleO",  this  function  calculates  the  z-axis  rotation  angle  to  align  the  x axis. 

M FD_mfd wi n d ow () : to  open  a MultiGen  window  as  the  working  MFD  for  editing. 

5. 2. 3. 7.4  File  mfddefault.c 

This  file  contains  several  routines  to  access  the  datapool  variables  and  return  their 
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MfD  mod  ctrl  bv  page():  This  function  is  called  by  the  page  editor  to  change  the 

Nlvnf^OS^Vana.b,eS  c?"tr°llab,C  by  VEST  such  35  scale  and  displa?  St  of 
NAV  pages.  The  values  of  these  variables  may  be  also  changed  by  the  simulation  models. 

5.2. 3. 7.5  File  mfddesk.c 

This  file  contains  functions  to  open  a desktop  window  which  shows  a set  of  library  tools 
and  allows  a user  to  select  a MFD  field  (tool)  to  be  added  to  a MFD  page.  ^ 

M FD_  bu  1 1 d_tooI_  win  do  w () : This  function  specifies  the  number  of  tools  in  the 
Whenever  a new  tool  is  added  to  the  library,  this  number  should  be  corrected 
accordingly.  The  tool  name  should  also  be  added  to  the  file  "mfd.msg". 

5. 2.3. 7. 6 File  mfddisp.c 

This  file  contains  functions  to  invoke  drawing  routines  of  MFD  fields. 

^MDTdrar-iPiCuU^e():  This/uLncti?n  controls  the  display  of  a MFD  page.  To  draw  a 
Cie|d’  "J™  chfclfs.lts  type  and  then  invokes  the  corresponding  drawing  procedure  for  the 
field.  If  an  underlying  page  is  specified,  it  draws  it  first  by  calling  itself  recursively. 

S-PiC.kfield():  The  functi?n  is  similar  to  the  function  "MFD_draw_picture()"  except 
IhfmouSTs^L^a't5  SCt  m **  P,Cking  m0de'  R iS  aSkd  10  dctcrmine  *e  field  which  ? 

5.2.3. 7.7  File  mfdfield.c 

This  file  contains  generic  functions  to  add  a field  to  a MFD  page. 

S°-!leld-fUr;CS():  ^ function  Provides  a way  to  allow  MFD  tools  to  create  their 
dialog  boxes  and  set  up  those  control  functions  when  "OK"  "CANCEL"  and  "TRY" 
buttons  are  pressed.  ’ ’ 

MFD  editfieldO:  This  function  is  called  by  MFD  tools  to  allocate  enough  memory  to 
store  the  attributes  of  a field  and  insert  it  into  the  linking  list  of  the  editing  page.  ^ 

^ -S  funcdon  controls  the  mouse  event  when  a field  is  being 

mn  fi  i!f  h ? mOUSe  Vs  preSseLd>  11  positions  the  field  to  the  cursor  point.  If  the  Middle 

ouse  is  pressed,  it  puts  the  graphics  state  in  the  picking  mode  and  the  attributes  of  the 

picked  field  are  copied  to  those  of  the  working  field.  armouiesor  the 

5.2.3. 7.8  File  mfdio.c 

™S  S®  contains  functions  to  save/open  MFD  database.  From  the  "filename"  given  by  the 
user,  the  I/O  routines  open  two  files  "filename.button"  and  "filename.tree".  lie  file  y 
i ename.button  is  a text  file  containing  information  about  initial  pages  of  MFDs.  The  file 
filename.tree  is  a binary  file  storing  the  MFD  page  structure  and  its  fields. 

MFD_save_db():  This  function  saves  the  MFD  database  in  memory  into  disk  files. 

Mroopen  db():  This  function  reads  the  MFD  database  files  into  the  memory  The 
reading  procedure  recursively  calls  itself  to  read  in  the  MFD  tree. 

5.2.3.7.9  File  mfdmain.c 
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This  file  contains  functions  to  interact  with  MultiGen®  Interface  Manager  and  allocate/free 
memory  space  for  a MFD  tree. 

MFD_init():  This  function  is  called  by  MultiGen®  to  set  up  the  menu  functions  of  MFD. 
It  also  sets  up  a text  window  for  page  mappings. 

MFD  menu():  This  function  handles  the  menu  events. 

MFD_mkbutton():  This  function  allows  other  functions  to  initialize  the  values  of  radio 
buttons  in  a MultiGen  dialog  box. 

MFD  mkcheckboxQ:  This  function  allows  other  functions  to  initialize  the  values  of 
check~boxes  in  a MultiGen®  dialog  box. 


5.2.3.7.10  File  mfdmod.c 

This  file  contains  functions  to  select  a field  in  the  editing  page. 

MFD  mod  field():  This  function  is  called  by  the  page  editor.  It  sets  up  a dialog  box 
and  puts  the  system  in  the  picking  mode.  It  controls  the  functions to  delete • a field,  o 
change  the  display  priority  (order),  and  to  invoke  the  corresponding  MFD  tool  to  modify  its 

attributes. 


5.2.3.7.11  File  mfdpage.c 


This  file  contains  functions  to  create/edit  MFD  pages. 


MFD  Dace  window():  This  function  sets  up  a dialog  box  and  defines  its  control 
functions  for  editing.  The  control  function  for  creating  fields  checks  the  MFD  tool  selected 
in  the  desktop  window  to  invoke  the  right  tool  functions. 

MFD  new  page():  This  function  creates  a page  structure,  inserts  it  in  the  tree  under  the 
parent” node? defines  the  default  attributes  for  the  new  page,  and  then  calls  the  function 
”MFD_page_window()". 

MFD  mod  page():  This  function  loads  the  attributes  of  the  selected  MFD  page  in  the 

editing  buffer  and  then  calls  the  function  "MFD_page_window()  to  modify  the  content  of 
the  page. 

MFD  duplicate  page():  This  function  creates  a page  structure  which  copies  the  content 

and  attributes  of  the  selected  page,  inserts  it  under  the  parent  node,  and  then  calls  the 
function  ”MFD_page_window()". 


MFD  delete  page():  This  function  tries  to  remove  a MFD  page  from  the  tree.  If  there 
is  anychild  page  under  it,  it  does  nothing  and  reports  an  error. 

MFD  tool  button():  When  a mouse  is  pressed  in  the  MFD  desktop  tool  window,  this 
function  is  called  to  determine  which  library  tool  is  to  be  invoked. 


5.2.3.7.12  File  mfdsprintf.c 

This  file  contains  functions  to  parse  user’s  inputs  for  dynamic  strings. 
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MFD_spnntf():  This  function  parses  a character  string  from  user  input  If  there  is  anv 
dynamic  portion  of  the  input,  it  is  converted  into  the  string  format.  The  dyi^mic  fiddTs  a 
datapool-vanable  id  m the  input  string  which  is  enclosed  by  '<’  and 

5.2,3.7.13  File  mfdstruc.c 

This  file  contains  functions  to  draw  the  MFD  tree  and  to  modify  its  structure. 

^LD^rfd^W-StT();  ThiS  fu"Cti0"  draws  the  ^ tree  in  a MultiGen  window  (tree 
winoow).  This  window  is  created  with  scrollable  bars  and  its  use  is  similar  to  the 
MultiGen  Structure  window.  The  drawing  steps  are  also  similar  to  the  MultiGen 
runcnon  ISD_drawstruc()". 

MFP-S*r uc_pick():  This  function  is  called  when  a mouse  is  pressed  inside  the  MFD 

tree  window.  To  save  memory,  the  picking  method  is  different  from  MultiGen.  It  sets 

a f,  s!ate  in  tbe  Peking  mode  to  load  the  selected  page  box  into  a picking  buffer 

^e  pkktng1  buffer  rOCCSS  15  ^ * SCa"S  the  ^ a«ain  to  fuid  the  page  loaded  in 

MFD  find  treenode():  This  function  scans  the  MFD  tree  to  find  the  page  whose  id 
matches  the  input  string.  v e m 

5.2.3.7.14  Other  Files 

Most  of  these  files  contain  MFD  library  tools  used  to  add  fields  to  MFD  pages  A new 
tool  can  be  easily  added  to  the  library  by  the  following  template: 

«“nWs°o/SfierdWind0W<):  ™S  funC'i0"  is  10  °pen  a dia'°* "O’1  K <he 
MFD_new_ToolName():  This  function  is  to  specify  the  default  attributes  for  the  field. 
mtributest-TOOlName-ParamS^:  This  function  is  to  Process  user  inputs  for  the 

^ck»ISe,eCt-T00,Name^:  ThiS  function  is  to  set  the  attributes  when  the  field  is 

Sdmf°rnSS0:  ™s  f"nc,ion  is  10  ,he  araibutts  of  fi'ld  »h‘"  ” <* 

MFD_dup_ToolName():  This  function  is  o duplicate  attributes  of  the  field. 
MFD_draw_ToolName():  This  function  is  to  draw  the  image  of  the  field. 

5. 2.3.8  Jack  Interface 

Jack  is  a software  package  to  observe  how  a human  mannequin  performs  a task  The 
(Vision1 4 £?gn  °fJackis  d^ribed  in  UPENN's  manuals.  Jack  User’s  Guide 

w” ,ack  (Dec  21  ■ ,988)- ,n  ^ docameni’ 
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The  figure  below  shows  the  design  structure  of  VEST-Jack  interface: 


VEST -Jack  Interface 


Figure  7. 

1“  *e  MultiGen  display 

attempting  t ®t  interact  with  Jack  except  the  reach  movement,  it  depends 

iZn",  SZSStZ  Sack  dfpfay  taaion  should  be  invotak  Before  this  tacn» 
is  called8,  VEST  modifies  the  GL  matrix  stack  by  the  interface  matrix  and  Ihen  reslores  t 
matrix  stack  after  the  drawing. 

When  VEST  receives  a reach  command,  it  puts  it  in  a constrain  table.  For  each  tick,  Y^ST 
foHowsHtts  lawand  emulates  the  location  that  the  movement  can  reach  at.  It  then  three 
Jack  to  reach  th^point  Before  a reach  can  be  performed,  the  MultiGen®  coordinates  must 
be  converted  intoPJack  systems  through  the  use  of  the  interface  matrix.  When  a reach  is 
iTsued  at  a MFD  button,  this  information  is  also  stored.  When  the  reach  is  completed,  it 
invokes  the  page-switch  function  of  MFD. 

5. 2.3.8. 1 File  a3i/a3ijcl.c 

This  file  contains  most  of  the  procedures  for  VEST-Jack  interface. 

TCL  draw  picture!):  This  function  controls  the  drawing  of  Jack  figures  and  eye 

£i«dl T^re anv  drawing  the  interface  matrix  is  loaded.  If  the  window  is  to  show  the 

drawn.  If  the  f.g»n=  in  drawn,  die  gmphical  objects  for 

the  eyes  may  obscure  the  cockpit. 

jCL  Mg2point():  Given  a MultiGen  point,  this  function  calculates  the  corresponding 
location  in  Jack  coordinate  systems. 

ICL  setuoO-  This  function  is  called  by  MultiGen®  Interface  Manager  to  pop  up  a dialog 
box  to  interactively  scale  and  move  a Jack  figure  so  that  it  looks  correct  in  the  MultiGen 

window. 
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JCLprocess  paraml  ():  This  function  processes  the  content  in  the  parameter  window 
to  calculate  the  VEST-Jack  interface  matrix.  See  the  Appendix  for  the  data  format. 

JCL_menu():  The  function  handles  the  menu  events. 

JCL_init():  This  function  is  called  by  MultiGen  to  initialize  the  VEST-Jack  interface. 

JCL  update  eycview():  This  function  is  called  by  the  VIEWS  module  when  a 
MultiGen  window  is  set  to  the  eye  view  of  a Jack  figure.  This  function  assigns  the  middle 
point  of  the  Jack  figure  as  the  camera  s position  and  the  vector  from  this  point  to  the  focus 
point  as  the  camera  s orientation. 

JCL  do  ani  mo ve() ; This  function  goes  through  the  constrain  table  and  performs  all 
unfinished  reaches  (for  one-tick  period).  If  a reach  is  marked  as  done  but  its  target  is 
marked  as  attachable  (such  as  control  sticks),  the  reach  is  always  issued  to  Jack  in  case  that 
the  object  changes  its  position  and/or  orientation  as  the  simulation  proceeds. 

JCL  fishish_allmove():  This  function  is  called  in  the  standalone  animation  to  finish  all 
reaches  m the  constraint  table. 

JCL  reset_ani  move():  Any  reach  request  first  calls  this  function.  The  request  is  put 
in  the  constraint  table  and  overrides  the  old  content  which  has  the  same  reach  id.  If  the 
target  is  a MFD  soft  button,  a flag  is  set  so  that  at  the  end  of  the  reach,  the  page-switch 
function  of  MFD  can  be  invoked.  v a 

JCL^MgReach () : This  function  controls  static  reaches  for  MultiGen  surfaces  bv 
popping  up  a dialog  box.  y 

JCL  new _eyetrace().  This  function  stores  the  positions  of  the  eyes  for  a Jack  figure 

SSJST'  * used  ,0  diaw  ,he  eye  mcm  and  10  deKrmin'  •*« 

^L.uP,!i0tStatUiSP:rrThis  function  loads  the  status  of  both  hands  and  viewing  direction 
models  datapo°  buffer  so  that  the  network  procedures  can  send  them  to  the  simulation 

JCL_jpoint2MG().  Given  a Jack  point,  this  function  calculates  the  corresponding 
location  in  MultiGen  coordinate  systems.  F ® 

5.2. 3.8. 2 File  JACK/jclreach.c 

This  file  controls  Jack  reach  and  head-turning  movements.  Using  Fitt's  law,  it  calculates 
the  point  to  be  reached  at  for  the  tick.  It  then  issues  the  reach  command  to  Jack  The 
functions  are  taken  from  Jack  source  files  with  modifications. 

5. 2. 3. 8. 3 File  JACK/jclinit.c 

•?DC1  iDnf’rD"8  the  environment  file  of  Jack  if  the  environment  variables  "PEALIB"  and 
are  defined.  Since  VEST  does  not  load  Jack  environment,  its  purpose  is  to 
avoid  the  segmentation-fault  error  caused  by  the  peabody-parser  when  these  two  variables 
cire  not  denned. 

5.2.3.8.4  File  JACK/stamp.c 
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This  file  contains  those  dummy  routines  called  by  the  peabody-parser  and  psurf-parser,  but 
useless  in  VEST. 

5.2.3. 8.5  File  JACK/assign.c 

This  file  is  taken  from  Jack  source  file  pea^y/a'sign.c.  It  contains  those  dummy 
functions  for  Jack  attributes  which  are  disabled  in  VbS  1. 

5. 2.3. 8. 6 File  JACK/delete.c 

Ms  file  is  taken  from  Jack  source  file  peabody/delete  ,c.  A functatis  added  to  it  to 
remove  an  old  environment  when  VEST  opens  a new  Jack  environment. 

5. 2. 3.8.7  File  JACK/draw.c 

This  file  controls  the  drawing  functions  of  Jack  images.  It  disables s all  Jack  attributes. 

Most  drawing  routines  are  taken  from  Jack  source  file  psurf/draw.c. 

5. 2. 3.8.8  File  JACK/peaparse.y 

This  file  contains  the  peabody-parser.  It  is  modified  to  ignore  Jack  attributes. 

5. 2. 3.8.9  File  JACK/psurfparse.y 

This  file  contains  the  psurf-parser  and  is  a copy  of  Jack  source  file  psurf/rp.y. 

5.2.3.8.10  File  JACK/msg.c 

This  file  contains  the  methods  to  display  Jack  messages.  Since  VEST  has  its  own  user 
messages  ere  eithe?  ignored  or  redirected  to  the  shtndarf  output. 

5.2.4  External  Interface  Detailed  Design 

As  mentioned  before,  VEST  is  driven  by  the 

"redraw"  event  of  each  window  is  mostly  determined  by  the  VIEWS  module, 
decision  is  based  on  the  network  messages  from  the  Executive. 

5.2.4. 1 File  simdata.c 

This  file  precesses  network  messages.  When  a message  is  receiv^  it  ^wks  Ac  memge 
Arne  and  performs  data  conversion.  The  messages  can  be  a reach  for  a MultiGen  object,  a 
reach  for  a MFD  soft  button,  or  updated  values  for  datapool  variables. 

SIM  connect  to  host():  This  function  opens  a TCP/IP  network  socket  for  read  and 

write?  VEST  pTocesses  the  user’s  setup  and  determines  the  location  of  the  Executive. 

SIM  read  network():  This  function  uses  the  UNIX  system  call  "selectO"  todetermine 
if ! has  ^ved.  If  no  message  is  in  the  queue,  it  returns  immediately  so  that 

VEST  can  process  any  mouse  event. 

STM  handle  msg():  This  function  checks  the  message  type  and  determines  what  action 

fo  ^Tfom  irthe  message  is  to  update  datapool  variables,  its  body  content  is  copied  to  the 
datapool  buffer. 
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?^,d°-TeaCh():  Jhe  reLach,  ™essag.c  has  two  fields:  reach  id  and  target  site.  The  reach 
s an  index  into  the  reach  table  specified  by  the  user  in  the  parameter  file  for  Jack 

left  MFD  ITft?h/rStfCh)faCter  °f  *£?.  S?te  name  is  ’L’’  rest  is  the  button  name  for  the 
left  MFD  If  the  first  character  is  R , it  is  for  the  right  MFD.  If  the  first  character  is  'E' 

a.«S£5£“  OU,Sld'  *'  heliCopKr'  0,her"is''  itisa  MultiGen  object  moving  ' 

-S!a2? ,h'  avia,or's  su,us  if  *' is  availab1'  “d  a 

SIM  proc  scriptO:  This  function  reads  a script  file  and  processes  it  to  set  ud  MultiGen 
da  abases,  window  positions,  VIEWS  setups,  CDE  databases,  MFD  databSL  Ld  Jack 

secondSe  vaTue  ^ ^ flle haS  tW°  fields‘  ^ f,rst fie,d is  thc command  and  the 

5.2.5  Coding  and  Implementation  Notes 

mherTnTr^  fllC  °ft  ^7  ^ fr0m  one  miS  4D/GT  ^chine  may  not  run  well  on 
SSi  wT  ne  \In  ,order  **  Potable,  the  shared  libraries  should  be  used  in  the 
niffn  !h  erna  {- Mu  uGen®  convens  user  input  into  integral  numbers  and  may 
overflow  the  graphics  engine,  especially  on  the  personal  IRIS  machines.  If  the  screen 
shows  weird  images,  try  to  change  the  resolution  (COM  internal  resol)  in  the  file 

S?'Cht0  . r Tmber  However’ t0  make  the  conversion  process  accurate  this 
number  should  be  as  large  as  possible.  ’ m,s 
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6.0  USER'S  GUIDE 


6.1  OVERVIEW  OF  PURPOSE  AND  FUNCTIONS 

This  section  describes  how  to  use  VEST  software  and  its  ffigjT*-  B'f°" 

the  cursor  at  icons  and  menus  on  the  graphics  workstation  display. 


6.2  INSTALLATION  AND  INITIALIZATION 

The  following  procedure  shows  how  to  install  the  VEST  software. 


\ be  the  bonne  directory  of  ft. 

3.  xvo- and'waS  fa Sing  the  files.  A directory  structure  will  be 

created  as  follows: 


Figure  8.  VEST  Source  files 


4.  Tocreate  a new  executable  VEST,  VEST  ( 

M-DOPT_GT"  from  the  CFLAGS  in  the  Makefile. 

6.3  STARTUP  AND  TERMINATION 

To  bring  up  VEST,  change  the  directory  to  the  home  directory  of  VEST  and  type  mgvest  . 

AfteTthe  desktop  of  MultiGen  is  set  up!  the  user  may  choose  the  options  under  the  VO 

pull-down  menu^ to  open  an  existing  database,  create  a new  database,  or  quit 

6.4  FUNCTIONS  AND  THEIR  OPERATION 


6.4.1  Phase  IV  Demo 

The  following  is  a summary  of  step-by-step  procedures  for  running  VEST  standalone 
demonstration  through  a script  file. 

1.  Change  the  directory  to  the  home  directory  of  VEST. 
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dom*116  t*1C  Command  mSvest  -DEMO/demosim"  and  wait  until  the  automatic  setup  is 

3.  Close  the  top  window  which  shows  a moving  camera  view  for  integration  demo 

'xpos‘  *• COVOTd  dU,°* b~ ■ ™s 

5.  Click  on  the  Comm  Flag  command  of  the  CDE  menu. 

6.  Click  on  the  Link  Animate  command  of  the  CDE  menu  to  start  the  CDE  animation. 

7.  Press  the  mouse  three  times  to  stop  CDE  animation. 

8.  Click  on  the  Demo  Window  of  the  MFD  menu  to  bring  up  MFD  windows. 

y.  Click  on  the  Link  Animate  command  of  the  CDE  menu  to  simulate  MFD  dynamic  fields 
Press  the  mouse  once  to  stop  the  animation. 

window^  °n  thC  HidC  Window  command  of  the  MFD  menu  to  bring  up  a MFD  tree 

11.  Click  on  the  Animate  command  of  the  MFD  menu  to  page  through  the  MFD. 

12.  Bring  the  tree  window  and  then  MFD  window  to  the  top 

13.  Point  the  cursor  at  MFD  soft  buttons  and  press  the  mouse 

window)  Wind0W  and  Click  on  D0NE  button  of  thc  ^og  box  (animation 

15.  Quit  from  VEST. 

6.4.2  CDE  Menu 

Open  Coord.  File  opens  an  external  file  and  converts  its  data  into  MultiGen  format  The 
format  is  contained  in  Appendix.  1 ne 

P°PS  UP  a ™e"u  that  contains  a list  of  predefined  instruments.  Select  the 
desued  instrument  and  click  on  the  OK"  button.  A pop-up  dialog  box  will  aDDear  for 

In  HUtf-  C !hk  Pn  the  "S“°W"  button  and  then  follow  the  instructions  on  a control  window 
to  define  the  location  of  the  instrument.  wmuow 

Process  Link  computes  the  3D  animation  data  for  each  instrument  in  the  link  list  which  is 
created  by  the  Load  Link  File  and/or  Link  Parameters  commands.  Before  any  CDE 
animation,  this  command  should  be  issued.  ^ 

Load  Link  File  opens  a binary  link  file  and  appends  it  to  the  CDE  parameter  link  list. 

Save  Link  File  saves  the  current  CDE  parameter  link  list  into  a binary  file. 

^ up  dialog  to  define  the  animation  method  on  MultiGen 
objects  and  linking  parameters  for  animation.  The  parameter  ID  is  the  variable  ID  in  the 

JnfmS  d °f  t?ifJeX|  C ipbofard  ™dow  after  the  Datapool  Default  is  loaded.  For  die 
animation  method,  please  refer  to  Phase  III  CDE  CSCI.  The  string  display  is  an  interactive 

^_.It  ne,eds  in.Puts  for  the  ^rting  point  (a  MultiGen  vertex),  local  x vector  (a  MultiGen 
edge),  and  local  z vector  (a  MultiGen  polygon).  1 

Color  Palette  brings  up  the  CDE  color-palette  window.  The  paired  colors  are  designed 
0n/°ff  swltch  of  warning  lights  and  numerical  leds.  When  the  writemask 
flag  of  a CDE  instrument  is  set  TRUE  in  the  Link  Parameters,  the  unerasable  colors  can 
cover-up  the  erasable  colors.  This  property  is  to  provide  some  Mwindowing"-effect 
animation  for  instruments  such  as  ADI  and  Drum  Dial  gauges. 

Insert  Color  modifies  the  color  of  the  selected  MultiGen  object 

Modify  Color  modifies  the  RGB  of  the  selected  color  from  the  CDE  color  palette. 
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Mod  Attributes  allows  the  user  to  browse  through  the  CDE  link  list  and  opens  the  Link 
P.^mw£CTo modify  U,e  linking  parameters of  the  selected  tnsmmtent. 

Data  Format  O/P  allows  a user  to  convert  MultiGen  database  to  other  formats. 

i ink  Animate  opens  some  external  files  to  update  current  values  in  the  VEST  datapool 
Lid  thus  shows  CDE  standalone  simulation.  This  command  can  also  be  issued  for 
standalone  animation  of  MFD  dynamic  fields. 

Unlink  All  removes  the  CDE  link  list. 

Open  Setup  loads  the  named  file  into  the  setup  window  for  VIEWS  animation. 

Save  Setup  saves  the  content  of  the  setup  window  into  a file. 

Process  Setup  processes  the  content  of  the  setup  window  and  then  opens  MultiGen 
windows  for  VIEWS  animation. 

Setup  brings  up  a text  window  to  define  VIEWS  setups.  The  format  is  contained  in 
Appendix. 

Ethernet  opens  a network  socket  to  the  Simulation  Executive.  It  also  sets  the  Comm 
Flag. 

Reset  suspends  the  communication  process  and  also  resets  the  Comm  Flag. 

Hide  Window  toggles  between  showing  and  hiding  a MultiGen  window. 

Focus  View  shows  a blown-up  view  of  the  selected  MultiGen  polygonal  surface  in 
VIEWS  animation. 

Comm  Flag  sets  the  animation  flag  to  tell  the  display  module  of  MultiGen  to  call  CDE  or 
VIEWS  functions  for  animated  objects. 

DataPool  Default  loads  the  named  file  into  the  VEST  datapool.  The  file  "default"  in  the 
directory  "DEMO/DATA"  is  supplied.  Once  it  is  loaded,  the  user  may  refer  to  ltfor  th 
variableid  through  the  Clipboard  window  of  the  MultiGen  menu  funcuon  TEXT  . 

Simulation  Script  open  a script  file  to  set  up  MultiGen  windows  and  animations  for 
CDE,  VIEWS,  MFD  and  JACK.  This  is  designed  for  Phase  IV  demo. 

Track  Window  toggles  between  showing  and  hiding  the  MultiGen  track  window. 

Show/Hide  Path  toggles  between  showing  and  hiding  the  flight  path. 

6.4.3  MFD  Menu 

It  is  noted  that  most  of  the  time,  when  a MFD  dialog  box  pops  up,  it  disables  MultiGen 
menu  and  dialog-box  functions.  The  user  must  take  action  against  the  dialog  box  before 

issues  any  other  commands. 

MFD  Base  pops  up  dialog  boxes  to  define  MultiGen  objects  as  MFDs.  Each  MFD 
should  have  a unique  ID  between  1 and  10. 
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MFD  ID  defines  the  editing  MFD  for  New  Page  and  Mod  Page.  This  allows 
switch  MFDs  with  different  physical  sizes  such  as  Longbow  UFDs. 


a way  to 


windowh°W/Hlde  t08gleS  between  showing  and  hiding  MFD  display  in  the  top  MultiGen 


NuZ  P?8u  ^ Up  llbraryT-t001  windows  to  create  a MFD  page  which  is  inserted  as  a 
child  of  the  parent  page.  However,  if  no  working  MFD  is  defined,  this  function  does 
nothing.  A MFD  can  be  defined  by  the  command  MFD  Base  or  Open/New.  If  no 
parent  page  is  specified,  the  selected  page,  if  it  exists,  is  taken  as  the  parent  page. 

Mod  Page  allows  a user  to  modify  the  selected  MFD  page. 


Dup  Page  duplicates  a selected  MFD  page  which  is  inserted  as  a child  of  the  current 
parent  page.  The  command  also  invokes  Mod  Page  for  the  duplicated  page. 

Delete  Page  removes  the  displaying  page  of  the  editing  MFD  from  the  tree. 


Select  Page  searches  the  MFD  tree  and  tries  to  find  the  page  with  the  name  matching  the 

user  s input.  The  user  may  also  opens  the  tree  window  and  uses  the  mouse  to  Dick  the 
page.  K 


Attach  moves  the  selected  page  to  be  a child  of  the  current  parent  page. 

Detach  removes  the  selected  MFD  page  from  the  tree. 

Set  Parent  sets  the  selected  page  as  the  parent  page. 

rama i necH n° Appe ndtx * Wind°W  for  users  input  t0  set  up  MFD  animation.  The  format  is 


reject  aU\oS  page  MuldGen  window  t0  show  the  MFD  ffee-  A user  can  use  the  mouse  to 

Animate  allows  a user  to  page  through  the  MFD  tree.  The  dialog  box  allows  the  user  to 
enter  the  reach  id  for  Jack  and  a reach  for  a waypoint. 

Open/New  opens  the  MFD  database  files  created  from  the  same  MultiGen  objects. 

Save  saves  the  MFD  database  into  a file. 

Close  removes  the  MFD  database  from  the  memory. 

SePElL0£a^draW  *' MFD  Pa*K  ^ MuUi0en  >° 

Open/Old  loads  the  MFD  database  created  from  another  set  of  MultiGen  objects  Before 
diis  command  is  called,  the  MFD  IDs  should  have  been  defined  by  MFD  Base  or 
Open/New. 

6.4.4  JACK  Menu 


Page  E-26 


New  Env  opens  a Jack  environment  file  which  defines  names 

SS" «•  *ese  tw0  can  156 

defined  in  a script  file  through  the  command  Simulation  Script. 


Param  Window  brings  up  a text  window  for  defining  parameters  of  the  VEST/Jack 
interface.  The  format  is  contained  in  Appendix. 

Setup  pops  up  a dialog  box  to  interactively  refine  the  portion  and  scale  of  a Jack  figure. 

Reach  Face  performs  Jack  reaches  at  MuldGen  polygonal  surface.  The  user  may  do  a 
single  reach  or  multiple  (two)  reaches. 

Jack  Shore/Hide  toggles  between  showing  and  hiding  displays  of  Jack  figures  in  the  top 


window. 


Open  Param  loads  a file  into  the  parameter  window. 

Save  Param  saves  the  parameters  of  the  VEST/Jack  interface  into  a file. 

Process  computes  the  3D  relations  between  Jack  and  VEST  from  die  content  of  the 
parameter  window. 

Tracer  Show/Hide  toggles  between  showing  and  hiding  eye  tracers. 


6.5  ERROR  AND  WARNING  MESSAGES 


MultiGen  communicates  with  the  user  thi  ^^^ssageushig  the'^Last  Error  Msg"  option 
^der^tl^S'lOT(T  p^dwnmema.6  ^^^e  m^se  once°an^  the  error  -message  window 
goes  away. 


6.6  RECOVERY  STEPS 

A core-dump  file  is  produced  when  VEST  aborts  abnormally. 

7.0  ABBREVIATION  AND  ACRONYMS 

A3I  Army-NASA  Aircrew/Aircraft  Integration 

DMA  Defense  Mapping  Agency 

CDE  Cockpit  Display  Editor 

MFD  Multi  Function  Display 

UFD  Upper  Front  Display 

VEST  Visual  Editor  and  Simulation  Tool 


8.0  GLOSSARY 

9.0  NOTES 

9.1  MISCELLANEOUS 

9.2  LIMITATIONS 
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The  limitations  for  using  VEST  are: 


1.  All  graphical  objects  have  to  be  in  a single  MultiGen  database  file. 


1?ie/’etwork  interface  must  follow  the  data  structure  of  the  Executive 
the  instruments  of  the  lead  helicopter  can  be  animated. 


As  a result,  only 


3.  The  library  tools  of  MFD  are  not 
configuration. 


"soft"  enough  and  are  limited  to  the  APACHE  longbow 


4.  All  MFDs  share  the  same  database  and 
setups  such  scales  on  NAV  pages. 


global  variables  and  cannot  display  their  own 


5.  VEST  Jack  can  only  perform  the  reach  movement  and  head  turning. 
9.3  LESSONS  LEARNED 


HninnTif-  VEST* one  shou‘ 1(1  toy to  figure  out  what  functions  are  currently  available  In 

code  may  tec'™  “ mak' '°°ls  “ *"*■*■*«  •«  Possible.  If  not.' the 


9.4  FUTURE  DIRECTIONS 

Future  enhancements  should  consider  the  following  aspects: 
1.  Performance 


CDE/hlR^ntemaIfomm'^ou/dI^lj^uc^tasCmuchrasptMriWe!'tWOr*t  O^SS'S'S  into 

2.  Portability 

gSTT Cl0Sely  d<? 10  *'  data  a™***  of  the  simulation 
pmvided  for  dSafonvemfon  Sh°Uld  **  USed  and  ««"*>  f“"«i°»s  « 

3.  Flexibility 

The  capabilities  of  VEST  are  limited  by  the  set  of  built-in  tools  It  will  be  better  if  evt^mai 

^libraries. Pr°V,ded  A ~ lh“  objects  * *• 


10.0  APPENDICES 
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APPENDIX  a.  — VEST  FILE  FORMATS 
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£*************cde  coord  FILE  FORMAT  ************/ 
Getnconvert  (fpath) 

FILE  *fpath; 

int  i,  j,  k,l,  nclustcr,  nobject,  nface,  nvertex,  color 
float  x,  y,  z; 
char  dummy[80]; 

fscanf  ( fpath,  "%d",  &ncluster); 
for  (1=0;  l<ncluster;  1++) 


fscanf  ( fpath,  "%d",  &nobject); 
for  (i=0;  i<nobject;  i++) 

fscanf  ( fpath,  "%d  %s",  &nface, 
for  (j=0;  j<nface;  j++) 


dummy); 


fscanf  (fpath,  ”%d  %d  %s",  &nvertex,  &color,  dummy) 
for  (k=0;  k<nvertex;  k++) 
fscanf  (fpath,  ”%f%f %f\  &x,  &y,  &z); 
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/******  VIEWS  setup  format  ****************/ 


'OBJECT  SECTION 

1 LEAD  16.0  16.0  16.0-12.3  0 0 x 0 y 0 z 180 

2 WING  16.0  16.0  16.0  -12.3  0 0 x 0 y 0 z 180 

3 TRUCK  2.0  2.0  2.0  0 0 0 x 0 y 0 z 0 
#starfish  1039  1041 

1 1 -14.5  5.5  15.0  x 0 y 6 z -45  7 1 pilot 

2 1 -14.5  5.5  15.0  x 0 y 6 z -45  7 1 pilot 


ed  by  descriptions  of  the  MultiGen  objects  The«  objects 
are  animated  by  their  locations  and  orientations  as  sent  by  the  Executive.  Note  that  the 
rotation  angles  must  be  specified  in  such  a way  that  the  heading  points  to  the  positive  x 
direction  and  the  top  points  to  the  positive  z direction. 


Filed  1 = Object  ID  as  is  defined  by  the  Executive. 
Filed  2 = MultiGen  Name  for  the  Object. 

Filed  3 = X scale. 

Filed  4 = Y scale. 

Filed  5 = Z scale. 

Filed  6 = X offset  from  the  terrain  origin. 

Filed  7 = Y offset  from  the  terrain  origin. 

Filed  8 = Z offset  from  the  terrain  origin. 

Filed  9 = axis  of  rotation. 

Filed  10  = degree  of  rotation. 

Filed  1 1 = axis  of  rotation. 

Filed  12  = degree  of  rotation. 

Filed  13  = axis  of  rotation. 

Filed  14  = degree  of  rotation. 


stSts'wi  tlf  "T1  followed  by  the  host-name  of  the  Executive,  socket  port  of  the  Executive 
and  own  socket  port.  The  section  body  has  14  fields: 


Field  1 = Window  ID. 

Field  2 = Object  ID  to  which  the  camera  is  attached. 
Field  3 = X offset  of  the  camera  from  the  object  origin. 
Field  4 = Y offset  of  the  camera  from  the  object  origin. 
Field  5 = Z offset  of  the  camera  from  the  object  origin. 
Field  6 = rotation  axis  for  the  next  field. 

Field  7 = rotation  degree  of  the  camera. 

Field  8 = rotation  axis  for  the  next  field. 


Field 

Field 

Field 

Field 


Field 

Field 


: rotation  degree  of  the  camera. 

= rotation  axis  for  the  next  field. 

= rotation  degree  of  the  camera. 

= an  ORed  flag  for  the  window  type. 

JACK_FLAG  (=1)  displays  JACK  figures. 

MFD_FLAG  (=2)  displays  MFD  pages. 

TRACERJFLAG  (=4)  displays  JACK  eye  tracers. 
EYEVIEW_FLAG  (=8)  attaches  the  camera  to  a JACK  figure. 

= flying  speed  for  interactively  adjusting  camera  positions 
l a way  similar  to  change  of  viewer-position  in  MultiGem 
* 2s  of  JACK  figure  needed  when  EYEVIEW.FLAG  is  set. 
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/*****  MFD  Mapping  format  ******************/ 

! 

1 CXDMMTOP 
2NAVTOP 

3 UFD 

4 CKYBD 

INITIAL  PAGE  SECTION 
It  starts  with  "!"  and  is  followed  by  descriptions  of 
the  page  names. 

Filed  1 = MFD  ID. 

Filed  2 = name  of  the  Initial  Displaying  Page  on  the  MFD. 


/*****  jack  Parameters  format  ******************/ 
.'MOVE 

1 pilot.left_fingers.distal  pilot. waist  2 

2 pilot.right_fingers.distal  pilot. waist  2 
+ATTACH 

ccoll 
ccyclic 
#pilot  1 1 2 
C 133 
F 0.076068 

S  20.000000  20.000000  10.000000  50.000000 
R z 1800  x -900  y 0 
P PCHAIR1 

M -0.823516  -2.331641  -2.268802 

1 1 MFD 

1 2 ccoll 


MOVE  SECTION 

This  section  starts  with  "!"  and  contains  the  lookup  table  for  a reach  ID  sent  by  the 
Symbolics  models. 


Field  1 = reach  ID.  Note  that  a reach  ID  between  20  and  29  is 
reserved  for  head/eye  turning.  The  symbolics  models 
uses  this  range  for  CPG.  The  VIEWS  module  which  controls 
the  network  communication  takes  of  the  difference. 

Field  2 = end  effect  of  the  reach. 

Field  3 = end  joint  of  the  reach. 

Field  4 = a flag  to  determine  whether  to  turn  head  as  the  reach  is 
performed.  When  the  value  is  0,  there  is  no  change  in 
eye  focus.  When  the  value  is  1,  the  focus  of  the  eyes 
is  set  to  the  target.  When  the  value  is  2,  the  head  is 
also  turned  to  the  target. 
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ATTACH  SECTION 


Thic  oftction  starts  with  "+"  and  takes  only  one  field.  The  field  is  the  MultiGen 
wSchSe  end  effect  of  a JACK  reach  is  constrained  and  attached  to  after  the  reach  is  done. 


and  defines  die  VESTOACK  m.erface  and  odiers.  Following 
it  takes  4 fields  as  follows: 

Field  1 = a string  to  identify  a JACK  figure  in  the  environment  file. 

Field  2 = a flag  to  identify  the  JACK  figure  as  the  pilot  or  CPG. 

When  the  flag  is  0,  no  network  status  is  sent  to  the 
Symbolics  for  this  figure.  When  the  value  is  1*  a CPU 
status  is  sent  out  When  the  flag  is  2,  a pilot  status 
is  sent  to  the  Symbolics. 

Field  3 = any  reach  id  associated  with  the  left  hand. 

Field  4 = any  reach  id  associated  with  the  right  hand. 


The  rest  of  this  section  defines  the  interface,  color,  and  initial  positions: 

*C  defines  the  color  index  for  the  figure. 

'F'  defines  the  scale  for  the  figure.  For  a one-to-one  mapping, 
this  value  is  1/12  since  one  interface  unit  is  12  JACK  units. 

T defines  the  initial  position  of  the  figure.  Field  1 is  the  reach  id 
and  Field  2 is  the  target  (MultiGen  object). 

’M’  defines  the  offset  of  the  JACK  figure  from  the  seat.  Note  that  one 

interface  unit  is  12  JACK  units.  . _ , . * 

'F  defines  the  seat  for  the  JACK  figure,  which  is  a MultiGen  objwt. 
'R'  defines  the  orientation  of  JACK.  Its  format  is  axis  degree  axis 
degree  axis  degree".  Note  that  the  degree  is  expressed  in  tenths. 

'S'  defines  the  incremental  speeds  in  three  axes  for  interactively 
adjusting  the  JACK  position  when  Setup  Menu  function  is  invoked. 
Note  that  one  interface  unit  is  12  JACK  units. 
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' PHASE  IV: 


DISPLAY  LAYOUT  ANALYSIS 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 


and  Analysis  System. 

1.2  SCOPE  OF  DOCUMENT 

The  Display  Layout  Analysis  module  was  developed  rapidly  to  demonstrate  proof  of 
concept  at  the  Phase  IV  A3I  demonstrations.  Accordingly,  the  emphasis  m this  dwument 
is  on  die  purposes  and  concepts  of  this  tool,  methods  and  approaches,  smd  apreliminsuy 
desien  Broadly  stated  the  Display  Layout  Analysis  module  explores  the  idea  of  assisting 
human-machine  interface  designers  with  a computer-based  tool  which  incorporates .human 
factors  design  knowledge  concerning  spatial  location  of  information  to  be  displayed  to 
humZ  oSorsof  the  machine.  This  document  is  directed  toward  three  categories  of 
readers'  T)  readers  with  an  interest  in  this  subject  and  issues  involved  in  developing  such  a 
tool  2)  readers  who  wish  to  learn  what  the  Display  Layout  Analysis  tool  s ^ent^id 
future  capabilities  are  and  3)  those  who  wish  to  use  the  tool  in  its  preliminary  fom  and 
exp^  C and  CLIPS,  an  expert  system  building  tool 

wntten  in  C at  Johnson  Space  Center,  is  helpful  but  not  necessary. 

1.3  PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 

This  document  provides  a description  of  the  putpose  and  approach of The  Uye« 

Analysis  module  for  the  reader  interested  in  an  overview  as  well  as  a description  of  the 
module’s  preliminary  design.  Also  included  is  a discussion  of  issues  and  questions  that 
have  come  up  thus  far.  A user’s  guide  is  available  to  assist  in  use  and  exploration  of  t 

tool. 


2.0  RELATED  DOCUMENTATION 


2.1  APPLICABLE  DOCUMENTS 

CLIPS  Reference  Manual,  Artificial  Intelligence  Section,  Johnson  Space  Center,  May 
1989. 


Giarrantano,  Joseph  C.,  CLIPS  User's  Guide,  Artificial  Intelligence  Section,  Johnson 
Space  Center,  June  1989. 


Hartley,  Craig  S.  and  Rice,  John  R„  "A  Desktop  Expert t System; as  a i Human  Factors  Work 
Aid",  Proceedings  of  the  Human  Factors  Society,  31st  Annual  Meeting,  1987. 
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Haskell,  Ian  D.,  Wickens,  Christopher  D.  and  Samo,  Kenneth,  "Quantifying  Stimulus- 
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Card,  Hochberg,  and  Huey,  National  Academy  Press,  Washington,  D.C.,  1989. 


Witken,  Andrew,  and  Kass,  Michael, 
22:159-168,  1988. 


"Spacetime  Constraints",  Computer  Graphics, 


Woods,  David  D„  and  Eastman,  Mary  Claire,  "Integrating  Principles  for  Human- 
Computer  Interaction  into  the  Design  Process:  Heterarchically  Organized  Principles"  in 
Proceedings  for  1989  IEEE  International  Conference  on  Systems,  Man,  and  Cybernetics, 
November  14-17,  1989,  Cambridge,  Mass. 


2.2  INFORMATION  DOCUMENTS 


The  following  references  were  not  used  directly  in  the  Phase  IV  version  of  the  Display 
Layout  Analysis  tool  but  are  related  to  this  general  problem  area  and  may  be  useful  in  future 


Beshers,  Clifford  M.,  and  Feiner,  Steven,  "Scope:  Automated  Generation  of  Graphical 
Interfaces  , Proceedings  of  the  ACM  S1GGRAPH  Symposium  on  User  Interface  Software 
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Hix  Deborah,  "A  Procedure  for  Evaluating  Human-Computer  Interface  Development 
Tools  . Proceedings  of  the  A CM  SIGGRAPH  Symposium  on  User  Interface  Software  and 
Technology,  Williamsburg,  Virginia,  Nov.  13-15,  1989. 


^Graphics'lruerfa  ^98*5  ^ ^raP^‘cs  ^nterface  Tool  Development",  Proceedings  of 
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Proceedings  of  Graphics  Interface/Vision  Interface  1986. 
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3.0  CONCEPT 

3.1  DEFINITION  OF  DISPLAY  LAYOUT  ANALYSIS  TOOL 

3.1.1  Purpose  and  Scope 

Design  of  the  human-machine  interface  for  use  in  complex,  dynanuc  environments  is  a 
fomudable  task.  A good  design  will  assist  the  human  operator  s performance  and  a poo 
design  will  hinder  it.  A prototypical  version  of  a Display  Layout  Analysis  tool  was 
developed  to  explore  the  concept  of  using  a computer-based  tool  with  human  factora  design 
knowledge  to  guide  the  process  of  designing  the  human-machine  interface.  Knowledg 
human-machine  interface  design  can  range  from  detailed,  low-level  guidelines  such  as 
display  luminance  and  font  sizes  to  higher  level  issues  such  as  placement  of  displays  in  the 
overall  design  configuration.  As  its  name  suggests,  the  Display  Layout  Atjjjtyu  .» 
focussed  on  assisting  the  designer  in  the  placement  of  displays  which  provide  informatio 
to  the  operator  from  various  information  sources. 

This  tool  currently  addresses  the  domain  of  aircraft  crewstation  display  panel  design. 
However,  it  is  expected  that  the  methodology  and  indeed  many  of  the  same  design 
guidelines  and  constraints  could  also  be  applied  to  human-machine  interface  design  in  other 
domains,  such  as  wind  tunnel  display  and  control  panels. 

In  designing  crewstation  display  panels,  a number  of  issues  arise. 

1)  What  information  does  the  pilot  need  for  effective  job  performance?  (e.g., 

2)  What  are  tfiesources  for  this  information?  (e.g.,  barometric  altimeter  and  radar 

altimeter,  airspeed  indicator)  . , , 

3)  If  we  consider  the  crewstation  display  area  to  be  the  design  space,  what  display 
media  (which  may  use  different  modalities)  are  available  to  be  placed  in  this 

4)  Howcan  displays  for  information  sources  be  grouped  or  used  singly  and 

allocated  to  the  various  available  display  media?  , 

5)  How  can  these  media,  which  provide  different  types  of  display  surfaces,  be 
located  in  the  design  space? 

The  emphasis  in  the  Phase  IV  demonstration  tool  is  on  assisting  the  designer  vrith  the 
simplest  case-that  is,  placing  dedicated  displays  on  the  control  panel.  However  a 
conceptual  framework  has  been  developed  to  address  the  more  general  cases,  wJiich  use 
advanced  display  media  such  as  Video  Display  Units  (\TDUs)  Mulu-Function  Displays 
(MFDs),  Heads-Up  Displays  (HUDs),  Helmet  Mounted  Displays  (HMDs)  Audl° 
Displays  (of  type  tone  or  message),  Tactile  Displays  (e.g.,  switch),  and  others.  Windows 


Page  F-3 


SubHrsimKlia)  appear  on  ,he  screen  for DLA  «** *> 

3*1.2  Goals  and  Objectives 

Eyidenee  supports  the  validity  of  the  human  factors  design  metrics  described  in  the  NRC 
report  (see  the  Section  on  Display  Layout  Analysis).  Our  goal  was  to  test  the  feasibility  of 
AS1Jhg  ^hSC  h,uman  factor.s  design  metrics  to  guide  the  designer  in  the  area  of  display  layout 
A short  development  period  of  about  6 months  led  to  rapid  prototyping  of  a demonstration 
tool  to  investigate  the  following  questions:  ui  a demonstration 

1)  Can  we  build  a design-aiding  tool  which  uses  the  proposed  human  factors 
design  metrics  to  guide  the  process  of  designing  display  layout7 
i)  Can  data  for  these  metrics  be  reliably  gathered? 

3)  What  type  of  graphical  user  interface  to  the  tool  would  best  assist  the  designers7 

4)  How  could  we  structure  the  tool  so  that  its  architecture  will  support  design 
^«^ce  in  definition  and  placement  of  advanced  display  media,  such  as 
displays?  DS’ HMDs’  as  we  1 as  traduional  display  layouts  with  dedicated 

Another  issue  was  to  investigate  using  two  approaches  in  combination,  a pro-active  analytic 
design-aiding  mode  and  an  evaluative  rule-based  advisory  mode. 

^aS^Vhe  eiT1Phasis  has  been  on  concept  exploration.  The  Display  Layout  Analysis 
tool  is  m the  beginning  stages  of  development;  a prototype  was  developed  quickly  to  tesf 

The  'h3e  general  Jasibl,lt>' and  utility  of  such  a tool  and  to  obtain  feedback  torn  attendees  of 
tne  AJi  Phase  IV  demonstrations. 

3.1.3  Approach 

After  Woods  and  Eastman  (1989),  we  consider  displays  to  be  viewing  windows  for  the 

KrnknSr°TUgh  Wh lfh  *f/s.he  °^tains  info™ation  about  machine  and  environment  status 
The  Display  Layout  Analysis  tool  assists  the  crewstation  designer  in  laying  out  the  disolavs 
which  provide  the  pilot  with  windows  to  the  information  sources  which  he/she  needs  to  ^ 
u cessfully  operate  the  aircraft.  Display  layout  assistance  is  guided  by  a set  of  design 
n mcorporated  into  the  tool,  which  can  be  operated  both  in  an  analytic,  design -aSng 
mode  and  in  an  evaluative  mode.  Just  as  the  spatial  locations  of  objects  in  toe  phScal 
world  are  completely  determined  by  the  forces  acting  upon  them,  the  locations  of^the 
information  sources  being  placed  by  designer  ate  delermined  by  the  human  factors-ielated 

SSfs“‘"®5 n"  *A*m;  ,n,‘h'  physical  worl<i- the  tees  am  entities  sucht  iSS 
Pi*  ”? lhe  %Play  ^yout  Analysis  tool  environment,  the  forces  whictfinfluence  the 
spatial  locations  of  displays  are  engineering  psychology  principles.  Based  on  the 
cumulative  forces  of  the  relevant  relations,  the  displays  can  be  relocated  to  optimize  the 
overall  design  with  respect  to  human  factors.  Used  in  the  evaluative  mode  die  tool 
wTmfnycSQhfnman  fnSineerLn£  desi.8n  guidelines  to  examine  an  existing  desi^  and  issue 
r™nFlT™lat,0nS  the.  guidelines.  A verbose  warning  mode  is  available  in  which  a 
ra  tonale  behind  the  warning  is  also  displayed  as  well  as  applicable  literature  references. 

TJie  human  factors  guidelines  currently  available  in  the  Display  Layout  Assistant  were 
obtained  from  the  section  on  Display  Layout  Analysis  by  Christopher  Wickens  in  Human 

Co£iTThe^fiHirr  C°mPMer-Aifad Engineering  (1989)  by  the  National  Research 
Council.  These  guidelines  are  described  below.  Estimates  for  how  strongly  display  pairs 
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are  related  to  each  other  in  various  dimensions  were  obtained  from  pilots  using 
questionnaires  as  was  other  display-related  information. 

physical  proximity,  ^ corolattonal  proxtmtty  for  the 
information  sources.  Definitions  for  these  measures  are. 

* jTsftefn^ 

of  system  flight  dynamics  (airspeed,  power  setting,  bank,  pitch,  etc.)  will  have 
rotor  functioning  are  more  similar  than  one  of  rotor  functioning  and  one  of 

. S&M73&  the  degme  » -** SSltT 

are  correlated  over  time.  For  example,  airspeed  and  power  would  De  mgruy 
correlated  but  bank  and  altitude  might  have  low  correlaoon  ofvduesthatis,  the 
aircraft  might  be  just  as  likely  to  be  level  as  banked,  in  low  and  high  flight. 

7)  According  to  symmetrical  location  compatibility  (one  forni  of  stimulus-response 

s^SffiSSSSSSSS&r- 

relevant  to  right-handed  controls  and  vice  versa. 

^ DiSDiavs  which  have  high  frequency  of  use  are  strongly  attracted  to  the  primes  on  ^ 

} ^s^lay  panel!  which  is  l "T  shaped  area  covering  the  top  of  the  panel  next  to  the 
windscreen  and  the  panel  center. 

With  advisory  assistance  from  Christopher fr^^o^fonhe^^an  faSJrs 

required  for  future  data  gathering  from  pilots. 

3,1,4  Description  of  Component  Architecture 

At  an  overview  level  of  description,  the  Display  Layout  Analysis  tool  consists  of: 

. functions  to  provide  a direct  manipulation  graphics  interface  “d,  viJ“f  “Om  methods  for 
assessing  design  merit;  also,  menus  to  provide  access  to  the  tool  s capabtltt.es  (user 

. SSS’SSL-  the  user  to  create  icons  and  edit  and  optimize  (manually  and 

! £ SSdS KS  advisor  to  check  an  existing  configuration  of  dtsplays  for 

. VSSfils SdmSalwttehim  factors  design  guidelines  (human  factors  data 
module) 
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a module  which  contains  geometric  data  and  equipment  models  (models  module) 
functions  to  save  and  recall  designs  in  progress  (housekeeping  module) 

All  modules  except  the  Advisor’s  rules  are  written  in  C;  the  advisor  rules  are  written  in  n tp<: 

£SS^riS3£:<a  Fig“re  1 


Figure  1. 


Overview  of  DLA  Tool  Structure  (Preliminary  Design) 


^play  Analysis  tool  is  currently  a stand-alone  module  constructed  to  exolore  the 
he  desLn nr^eT^ /actors  metrics  expressed  both  in  mathematical  and  heuristS  fomTto  ^tide 
with  " is  “kely  ‘ha' Uwil,te  ‘“f* 


3.2  USER  DEFINITION 


of  the  Display  Layout  Analysis  tool  are  expected  to  be  designers  of  human-machine 
wj10  want  to  take  into  account  human  factors  design  guidelines  1)  when  grouoine  and 
placing  displays  for  information  sources  on  the  various  tvoes  of  disnlav  «iirfnrv»c  aS  t?  u 
'““"8  display  media  in  the  design  space  “e?erSto  “ 

whic^fh^nil™  kV‘yCh,t  Anal/ils  1001 10  dt«cUy  manipulate  the  placjlnent  of  displays  through 
hich  the  pilot  is  to  obtain  information  about  environment  and  aircraft  status.  ^ “ 
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3 3 CAPABILITIES  AND  CHARACTERISTICS 

^teSESSSE***- 

tool,  the  designer  builds  a set  of  pages  by  g P ^^<.5  pages  can  be  collected  m 

aSS^wttsr- 

diagrammed  in  Figure  2. 


BUILD  THE  SET  OF  PAGE(S) 
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3a£“£3sMS«Sf5S^» 

AE^S^^,^^SfcgSta^^^g,SS^,,5£^5• ?,rcng,h  «!*  rc‘adon- Ate 

single  infonmation  source,  a set  of  sources  or  all  sources!  ™ d'sl*ner  can  show  relations  for  a 

Both  manual  and  automatic  optimization  modes  are  avaiiaKi*  xira , „ ... 

manually  by  showing  the  optimization  vector  for  a displiy  ^n  wW^TnSKfd^f^^ 
reduces  tension  from  other  related  sources  and  mm,;™  . cn  5°!nts, m “?c  mmction  that 

the  optimization  vector.  The  usercan  also’seleetT^  toward  the  location  indicated  by 

moves  displays  to  reduce  network  tension  for  the  |oup  htoth ^ 
display  icons  are  seen  graphicallv  A pinhai  nntmli,  .?'  DOtn  case?«  “?c  cffects  of  moving  the 
optimization  acconiingWS  Sc  ,ndlCa,K 

rationale  for  each  warning  as  well  as  any^^i^^refcrencMPuhe  human^w^  litoa^re8 
3.4  SAMPLE  OPERATIONAL  SCENARIOS 

from  various6 TOofSSw  sources  were  selected 

sources  which  would  be  * '0,al  m‘mb'r0, 

In  a typical  use  of  ihe  prololype  DLA  lool,  the  designer  would  perform  rhe  following  activities: 

' C«lt  V“°US  relad0ns  availab1'  (source-»“'-“.  source-panel,  souice-associated 

; i^SSSSS^ 

. posido"s in 

invoke  automatic  optimization  for  a selected  set  of  sources  and  ’ 

K die  for  deviations 

These  actions  are  described  in  more  detail  in  Section  6.0,  Users  Guide. 

4.0  REQUIREMENTS 

4.1  REQUIREMENTS  APPROACH  AND  TRADEOFFS 
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With  respect  to  requirements,  the  tool  should: 

. incorporate  valid  human  factors  design  guidelines 

I be  res^sivesScefl  tab^iued  to  assist  the  design  process 
! and  how  tochange  it  in  a way 

that  will  improve  the  design 

4 2 HARDWARE  ENVIRONMENT 

tool  is  expanded. 

4.3  SOFTWARE  ENVIRONMENT 

The  analytic  portion  of  the  tool  and  the  graphics  interface  are  written  in 

MSydirough  Friday  at  (713)  280-2231  Others  can  obmn  CLffS  for  about  $350  from 
COSMIC;  382  E Broad  St.;  Athens,  GA  30602;  phone.  (404)  542-3265. 

4.4  EXTERNAL  INTERFACE  REQUIREMENTS 

with  the  Cockpit  Display  Editor  (CDE)  and  other  equipment  models. 

The  DLA  tool  was  required  to  be  easy  to  use;  the  purpose  of  the  tool’s  interface: is  to  give : the 

capalsilides.'^he^w^hould^so^'veo'res^nsrv^o^^^^sfncc^low  response  time 
would  greatly  diminish  the  tool's  usefulness. 

4.5  REQUIREMENTS  SPECIFICATION 

4.5.1  Process  and  Data  Requirements 

information  sources  is  expanded. 

4.5.2  Performance  and  Quality  Engineering  Requirements 

c.  ra  a tool  is  to  be  used  in  the  design  process  for  manipulating  icons  in  the  design  space, 

it'isrequired'to  The  designer  needs  almost  instantaneous  response  „me. 

4.5.3  Implementation  Constraints 

Since  MIDAS  is  intended  to  be  distributed  in  whole,  or  in  parts,  to  the  user  community  at  a low 
cos^u^oHowVost  or  free-of-charge  tools  is  encouraged  to  avoid  ly  license  fees.  This 
requirement  had  an  impact  on  choice  of  the  expert  system  building  tool. 
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5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

5.1.1  Design  Approach  and  Tradeoffs 

the  *?um“  fc*ctors  design  metrics,  a questionnaire  (shown  in  Appendix  A)  was 
da*a  gadiered  fr?m  administering  the  questionnaire  formed  the  relations  between 
c/frt  3thS  WUh  ,each  other  and  Wlth  areas  of  the  design  space  and  with  associated  controls  For  a 
^ questi°tl"aire  was  given  to  two  AH-64  pilots  whose  responses  agreed  fairly  well  in 
some  areas  and  differed  widely  in  others.  To  be  valid,  a larger  sample  will  be  needed  in  future 
f°  ^ gathered  from  pilots;  however,  the  data  gathered  was  sufficient  to  build? 
f 1 S^!rnStrf0n  purposes-  Eventually  some  of  the  necessary  data  may  be 

j simulation  results.  Currently,  each  element  in  the  task  proximity  matrix  is 
computed  by  weighted  average  of  functional:  physical:  correlational  relations  data  using  a ratio  of 
2.1:1.  This  weighting  is  subject  to  future  change.  8 

Under  implementation  issues,  it  was  decided  that  the  DLA  tool  would  run  on  the  SGI  machine  to 
take  advantage  of  their  color  graphics  capabilities  and  to  be  compatible  with  the  Cockpit  Disolav 
Editor  (CDE),  with  which  the  DLA  tool  will  eventually  integrated.  Color  is  used  to  assist  the  y 

kcnin  a3nd  eYauatl°"  Proess;  for  example,  borders  of  display  icons  for  which  warnings  are 
issued  are  colored  red  to  indicate  a warning  message  applies  to  them.  g 

,^frrs,nHISme^ti°n  I8™?  expert  system  building  tools,  CLIPS  was  chosen  since  it  is  written 
m C and  will  run  on  the  SGI  machines  and  because  it  met  our  need  for  a proof  of  concent 

prototype.  It  was  decided  that  the  DLA  tool  should  reside  on  a single  maditoe  to  avoid  heavv 
network  penalties  during  run  time;  this  ruled  out  use  of  ART.  Also,  CLIPS  source  code  is  ^ 
available  to  government  users  at  nominal  cost  and  distribution  costs  are  lo^S  wUh  a 
commercial  expert  system  shell.  a 

£^J^0gram,desig?  (ease  of  maintenance)  dictated  the  need  for  a single  copy  of  the  human 
dors  design  datai  this  copy  is  used  by  both  the  analytic  and  the  rule-based  portions  of  the  tool. 

5,1.2  Architectural  Design  Description 

the.hu™"  facJors  guidelines  resides  in  matrices  and  vectors  and  is  accessed 
*hlLanal>'tlc  and  rule-based  portions  of  the  tool.  (CLIPS  rules  call  C functions  to  access 
ctnr^  a F“n.ctl0nal*  physical,  and  correlational  proximity  relations  between  display  pairs  are 
stored  in  matrices,  while  frequency  and  location  of  control  action  are  located  in  vectors  For  each 
display  pair,  a weighted  average  of  functional,  physical,  and  correlational  moS^TrelatiOTsTas 
IZytS  Pr°Ximity  rdati0n’  Which  forms  one  of  the  networks  us2dTn  desi^n 

Four  housekeeping  functions  include  file  operations  which  allow  the  user  to  save  designs  in 
progress  and  recall  them  later.  (These  are  currently  inactive.)  g 

The  design  functions,  which  are  to  be  used  in  close  conjunction  with  the  analysis  functions  allow 

P ' Jhe  analyze  funct'ons  allow  the  user  to  view  relations  between  displays 
relative  to  the  current  design  space;  these  relations  appear  as  colored  networks  with  color5  ^ 

which^nHiran type  °.f  rel?tion-  The  user.can  aJso  show  optimization  vectors  for  display  icons 
„ ch  mdicate  the  direction  and  force  with  which  an  information  source  is  pulled  There  is  an 
automatic  optimization  feature,  which  repositions  all  selected  display  icons  into  their  natural 
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"resting"  place,  given  the  attractor  and  repeller  forces  acting  on  them  and  tension  from  related 
information  sources. 


5.1.3  External  Interface  Design 

aaS8SEE=553S5Si5S& 

space  by  dragging  them  with  the  mouse. 

5.2  DETAILED  DESIGN 

5.2.1  Detailed  Design  Approach  and  Tradeoffs 
for  DLA  is  one  area  that  needs  to  investigated  further. 

m A „cpc  an  iterative  solver  for  a damped  oscillator  to  optimize  the  IS  locations.  This 

maximum  may  not  be  the  best  but  only  another  good  solution,  due  to  the  heunsnc  nature 
the  metrics.  The  pseudo-code  for  this  algorithm  follows: 


for  all  j do 

relation_vector  = X relation!], i]  * Distance[j,i]) 
i=l 

force.vector  = X force[i]  * Frequencylj]  *cofactor[i,j] 
i=l 

tension  = force_vector  + relation_vector 
step_dir  = compute_step_dir  ( j,  tension  ) 
if  unoccupied(  j,  step_dir)  then  move  ( j,step_dir) 

end  for  all  j 


3n  ISU1  and  ISU1.  DistanceU.il  is  the dntl « gg 

„nH  TQrn  Frenuencv  fil  is  the  relative  frequency  with  which  IS[j]  is  accessed  Dy  tne  pno 
ill'sk  jSXSe. eS.  IS  can  oil?  mo/e  one  step  par  iterauon  and  only  rf  them  » 

a vacant  cell  at  that  location. 


trend  of  the  solution. 

The  nntimizer  does  not  work  well  for  multimode  solutions  spaces  and  will  get  trapped  in 
Uxafmaximum^For  this  reason  it  is  suggested  that  a second,  more  cpu ' ““enstve, 
aUoriSTim patented  that  could  be  called  when  the  first  method  is  unsatisfactory. 
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5.2.2  Detailed  Design  Description 

5.2.2.1  Compilation  Unit 

The  following  source  files  comprise  the  DLA  tool: 

- ece.h 

- metncs.h 

- main.c 

- windows.c 

- menus.c 

- hmd_win.c 

- main_win.c 

- mfd_win.c 

- clips_win.c 

- draw_area.c 

- icon_win.c 

- creatc.c 

- metrics.c 

- task_prox.c 

- dla-fns.c 

- select.c 

- optimize.c 

5. 2. 2. 2 Detailed  Design  of  Compilation  Units 
Not  done  since  current  version  of  DLA  tool  is  a prototype. 

5.2.3  External  Interface  Detailed  Design 

Not  done  since  current  version  of  DLA  tool  is  a prototype. 

5.2.4  Coding  and  Implementation  Notes 

The  human  factors  design  data  describing  information  sources  and  their  relations  to  each  other  is 
stored  in  matrix  and  vector  form: 

3 2-D  arrays  for  functional,  physical,  and  correlational  relationships 

• 1 vector  for  frequency  of  use 

• 1 vector  for  location  of  control  action 

• 1 vector  for  criticality  (not  used  yet) 

The  contents  of  the  DLA  tool  Makefile  appear  below: 

# 

# makefile  ece 

# 

EXE  = ece 

#INCLUDEDIR  = /usr2/prevost2/panel/include 
#L1BDIR=  /usr2/prevost2/panel/lib 

#INCLUDES=  -1$  { INCLUDEDIR ) 

#HEADER=  ${INCLUDEDIR)/panel.h 
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LIB=  ../libclips.a  ../mfd/libmfd.a 

OBJ=  main.o  windows.o  menus.o  hmd_win.o  main_win.o  mfd_win.o\ 
clips_win.o  draw_arca.o  icon_win.o  create.o  mctrics.o  task_prox.o\ 
dla-fns.o  select.o  optimize.o 

SRC  = $(OBJ:.o=.c)  ece.h  Makefile 

#$(OBJ) : $(SRC) 

ece:$(OBJ)  Makefile  ece.h  metrics.h  . i 

$(CC)  -o  $(EXE)  $(LDFLAGS)  $(CFLAGS)  $(OBJ)  $(LIB)  -g  -lgl_s 
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6.0  USERS  GUIDE 

6.1  OVERVIEW  OF  PURPOSE  AND  FUNCTION 

The  Display  Layout  Analysis  program  is  a computer-aided  design  tool  with  built-in  human  factors 
knowledge  which  can  be  invoked  to  guide  the  design  process  aS  evaluate  dSnTta SSreST” 
with  respect  to  layout  of  displays.  The  tool  addresses  design  of  display  panels  Hr  man-machine 
systems,  the  cuirent  focus  is  on  rotorcraft  crewstation  display  panel  design.  The  tool  was 

¥ 3 de|?0^s&adon  version  using  rapid  prototyping  in  order  to  test  the  feasibility  of 
iSnJS  *Lth,  for  designers.  The  tool  is  in  the  beginning  stage  of  development.  Currently 
iSo^T-^  faCt0rS  dcs}$n  fuldehnes  (suggested  by  Christopher  Wickens  in  die 
o£fn?hnv 1989  iWhlCh  P®*31"10  display  layout;  more  are  envisioned  Hr  the  future.  A number 
wies  are/^SO  some  of  whlch  are  indicated  in  the  menu  selections  and  display 

media  windows.  Currently  dedicated  displays  are  emphasized;  in  the  future,  more  complex  P V 
display  media  and  surfaces  will  be  included,  such  as  VDUs  (Video  Display  Units)  MFDs  (Multi- 
Function  Displays),  HUDs  (Heads-Up  Displays),  and  HMDs  (Helmet  Mounted  Displays)  The 

S’.SS'T?  T in  l*®™1  enough  fettums  exta^vXuir^look 

and  feel  of  what  a human  factors  based  design-aiding  tool  might  provide.  The  planned  DLA 
design  process  is  shown  m Figure  3.  The  user  first  selects  one  or  more  displays  and  places  them 
on  a set  of  pages;  dedicated  displays  have  one  page,  while  MFDs  have  multiple  pagesP  The  set  of 
HMn  r^n1 P?fpped JP  a display  medium,  such  as  panel  (for  dedicated  display)!  MFD,  HUD  or 
final|y thl*  medium  is  mapped  to  a physical  coordinate  system.  Currently  dedicated 
316  emphasiz^ [although  other  display  surface  types  such  as  VDU,  MFD,  HUD  HMD 
and  audio  are  shown.  The  current  (demonstration)  version  of  the  tool  has  thirteen  pre-selected  ’ 

relSnaS^TCeS  °"i!he  PanA  T!,e  USer  can  movc  infonnation  soureS  a^n?  K 

fSAbSSch^ffife) is  deraibed  in  more  detail  *“  Fi8ure  2 “ 33 

SS S te !SSS te™.  mani';“la,i0"  “>  KS*"K  eventually  many  mom  WotLdon 

For  the  three  human  factors  design  metrics  currently  in  use,  data  includes  functional  physical 

matnces)>  frecJuency  of  use  (vector),  and  location  of  control 
of  ‘ P^f  ons  for  the*e  metncs  are  given  in  Sec.  3.1.3  (Approach).  A description 

iSsnsnssssr m Sec,ion  33  <capabmtiK  and  Jess, 

6.2  INSTALLATION  AND  INITIALIZATION 

The  required  DLA  program  files  are  ece  (for  ergonomic  crewstation  editor),  which  is  the 

S?P?f  e fde  for  the  DpA  tOQl.  and  two  data  files,  dla.clp  (contains  Advisor  facts  and  rules  in 
CLIPS  format)  and  dla-data.clp  (contains  Advisor  facts  in  CLIPS  format). 

6.3  STARTUP  AND  TERMINATION 

Sden^CT^5  and  thC  pr0gram  flles  for  DLA  1001  are  installed,  go  into  the  ../clips/ece  directoty 
% ece 
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A number  of  display  media  appear  as  windows  the^ 

IteKteen  and  stop  the  flicker.  A text  window  appears  below  fteDLA  panel  for  displaying 
messages  from  the  DLA  tool  and  communicating  with  the  Advisor. 

6 4 FUNCTIONS  AND  THEIR  OPERATION 

S£5sfes^.yjsa 

displays!  This  might  be  placed  in  the  lower  right  comer  if  the  pilot  is  expected  to  be  wearing 
monocle  over  the  right  eye,  for  example. 

aS^SSSSSSS^^r 

SSSSSSSfe? 
^JSSSESSSSs&s  t . 

have  the  action  take  effect.) 

* File  options:  These  include  Load,  Save,  Relations,  and  Others.  T^ey  are  n^  yet  active 
but  are  included  to  show  a method  of  saving  and  recalling  designs  in  progress. 

* “soume-Soume  hem  means  information  source  and  its  3 auf>optemu  «dw=: 

. Move:  Enter  "move  mode"  to  move  display  icons  around  the  design  space,  enter 

. NaiT^'mer^naiTmcxle''  to  fix  the  location  of  designated  displays;  enter  escape  to 
leave  nail  mode.  The  center  of  the  display  icon  turns  red  to  indicate  it  is  nailed 

. Revert:  Returns  all  displays  to  their  initial  location. 
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• Region:  Edit  region  is  inactive. 

• Relation:  The  edit  relation  items  (functional,  correlational,  and  physical)  arc  inactive. 
Create  options: 

Add  information  source  is  inactive,  but  will  be  used  to  bring  new  information  sources 
into  the  design  space. 

Source  items  are  inactive,  but  show  the  categories  of  information  sources  which  will 
be  available  in  a library  (Communication,  Navigation,  Engine,  Flight,  Hydraulic 
Environmental  Control  System  (ECS),  Electrical,  Auxiliary  Power  Unit  (APU) 
Lighting,  Target,  Other)  h 

• Region:  Add  region  is  inactive. 

• Relation:  Add  relation  is  inactive. 

Show  options  are  active: 

All  relations:  shows  task  proximity  network  and  frequency  network  for  all 
information  sources.  Task  proximity  connections  are  shown  in  purple;  attraction 
forces  of  sources  to  selected  spots  on  the  attractor  bars  are  shown  in  green  In  both 
cases,  the  strength  of  the  connection  is  indicated  by  line  thickness.  Use  Revert  to  turn 
these  networks  off. 

• Local  relations:  enter  "show  local"  mode;  clicking  left  on  a display  icon  causes  the 
task  proximity  network  for  that  information  source  to  be  shown  in  purple  to  indicate 
graphically  how  the  selected  information  source  is  related  to  other  sources  in  the 
design  space.  Enter  "escape"  to  leave  the  "show  local"  mode. 

Optimization  Vector:  enter  "show  optimization  vector"  mode;  to  show  a vector  for  any 
information  source,  simply  move  the  cursor  over  it  and  click  left  mouse  button  Leave 
this  mode  by  entering  escape. 

Tools  option  is  active  and  currently  contains  the  Advisor  and  optimize  options* 

• Advisor  capabilities  include:  y 

?fVrt1U|te:  th®  Adviser  on  the  current  design  configuration.  The  advisor  looks 

at  the  locations  of  all  displays,  checks  for  violations  of  the  human  factors  design 

in  ts  rul<Ts’  and  issues  applicable  warnings  in  the  text  window 
When  the  Advisor  is  invoked,  it  asks  "Print  rationale  and  references  for  warnings? 
(y/n)  , enter  y to  obtain  verbose  printout  mode. 

Reset:  reset  the  advisor.  Note:  the  Evaluate  option  also  resets  CLIPS  before 
performing  the  evaluation.  Having  a separate  Reset  option  allows  the  user  to  view 
Step)8enda  bef°re  eva  uation  and  t0  steP  the  agenda,  one  rule  at  a time  (sec 

• Show  Facts:  list  all  facts  currently  in  the  Advisor's  fact  base  in  the  text  window. 
Show  Agenda:  list  contents  of  the  agenda  in  the  text  window 

• Step;  Fire  one  rule  from  the  top  of  the  agenda;  used  in  debugging. 

Watch:  turns  on  the  "watch"  mode  for  facts,  activations,  compilations  (currently 
not  used),  or  all  of  these,  which  causes  the  relevant  information  to  be  printed  to  the 
text  window  when  the  Advisor  is  invoked. 

• Unwatch:  turns  off  "watch"  mode. 

Dribble  On:  start  a dribble  file  for  everything  printed  to  the  text  window.  Output 
to  the  text  window  is  also  written  to  the  dribble  file,  dla*clips-run  log 

• Dribble  Off:  turn  off  dribble  file.  v 6 

• Print  Rule:  display  a sample  rule;  demo  use  only. 

Optimize:  active;  provides  for  automatic  optimization  of  locations  of  selected 
information  sources. 

Local:  enter  the  optimize"  mode;  clicking  left  on  a display  icon  while  in  this  mode 

will  optimize  locations  of  all  icons  in  its  group.  Enter  "escape"  to  leave  this  mode 

• Group:  same  as  local. 
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• Global:  inactive. 


* Quit:  exit  from  the  DLA  tool. 


6.5  RECOVERY  STEPS 

If  the  DLA  tool  or  CLIPS  crashes,  start  over  by  entering  "cce"  into  the  text  window. 

7.0  ABBREVIATIONS  AND  ACRONYMS 


APU  - Auxiliary  Power  Unit 

DLA  - Display  Layout  Analysis 

ECE  - Ergonomic  Crewstation  Editor 

ECS  - Environmental  Control  System 

GL  - Graphics  Library 

HMD  - Helmet  Mounted  Display 

HUD  - Heads  Up  Display 

MFD  - Multi-Function  Display  ~ 

MIDAS  - Man-machine  Interface  Design  and  Analysis  System 

SGI  - Silicon  Graphics  Inc. 

VDU  - Video  Display  Unit 


8.0  NOTES 

8.1  ISSUES  AND  IDEAS 

A number  of  interesting  issues  and  questions  have  arisen  in  the  course  of  developing  this  tool. 

be  integrated  (if  at  all)? 
this  be  too  difficult? 

. Inter-  and  intra-media  information  transfers  and  relations  need  to  be  investigated. 

(e.g.,  varying  available  depth  behind  the  panel  at  various  locations). 

• asa?  asms  ss  - *«• 

of  fixed  wing  crewstation  panels  or  power  plant  display/control  panels? 

8.2  LIMITATIONS 

The  DLA  tool  is  limited  to  2-D. 


Page  F-17 


Currently  we  use  only  13  displays  which  is  less  than 
Apache  cockpit. 


one  fourth  of  the  displays  used  in  the 


‘Hfonption  for  the  task  proximity  measures  and  other  relations  must  be  acquired  and 
devices1" 1113111065  and  vectors-  Thcre  m no  metrics  currently  for  menu  spaces  in  multi-page 


8.3  FUTURE  DIRECTIONS 

Future  directions: 


Include  a more  comprehensive  set  of  information  sources  in  a library  (currently  13  are 
used).  Allow  user  to  define  new  information  sources. 

Incorporate  a richer  set  of  human  factors  principles  for  display  layout;  one  such  principle 
is  local  stimulus-response  compatibility.  y ^ 

Further  develop  both  analytic  mode  and  evaluation  mode  to  handle  more  display  media 
Current  focus  is  on  dedicated  displays  and  MFDs.  p y 

Enable  designer  to  take  into  account  more  contexts.  The  current  context  includes  a mix  of 
flight  and  commumcauon  tasks. 


Gather  data  from  more  pilots.  Current  task  proximity  measures  are  computed  using 
estimates  from  two  pilots.  6 

• Expand  capabilities  for  design  aiding  in  the  analytic  mode 

• Investigate  possibility  of  incoiporating  a prototype  Hypertext  capability  in  the  evaluation 


Iv«mnEat^nVirtral  reality  technol°gy-  Designers  can  work  in  a 3-D  environment  and.  for 

SSS5S  &eSma°0n  S°UrCeS  ^ HM° (HdmCt  Mountcd  Di$PIay>  ^inates 

Add  more  visualization  tools. 

Add  more  forces. 


Continue  literature  search  to  become  more  aware  of  related  work. 

* Cooperate  with  other  researchers  involved  in  complementary  areas 

• Allow  S !o  c£|eld  3d  “fKbehaVi0rS  and  Chlne'  WdShdn8S  for  exist“18  "'“ons 


9.0  APPENDIX 
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PILOT  QUESTIONNAIRE 
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Questionnaire  on  Crewstation  Displays 


We  seek  to  budd  a tool  to  help  the  A/C  designer  lay  out  displays  on  the  crewstation  panel  A trond 
layout  will  be  determined  by  how  the  displays  relate  to  each  other  and  their  relative  position  on  the 
panel.  Aircraft  displays  can  be  related  in  several  ways:  they  may  be  used  together  in  nerformino  » 
tosk  (functional  proximity);  their  information  can  come  from  the  same  or  sinular  physical  8 

’ the,enginf  (Phy^lcal  Proximity);  or  their  values  can  change  together  over  time  as 
fl  ght  processes  (correlational  proximity).  Together  these  measures  make  up  what  we  call  "task 

“,f°rmaUOn  fro”  exptns  such  as  ' » how  (and  wTetaJ  S Sc 

All  together,  there  are  four  matrices,  one  for  each  of  the  three  measures  of  task  proximity  and  one 

are  °^C°n,?ih  aCt,0n  ^:e'’  whlch  hand  do  y°u  use  to  control  the  subsystem  whore  values 

displays6  * P ^ 3 gIven  instrumcnt)-  This  questionnaire  uses  only  a subset  of  all  the  possible 

also  a 4^255”  ^ U e,lplai,Kd  a''hebonom  of“ch  PVf-  Them  £ 

K -i£;  - "p  „ 

to  perform  the  task  related  to  the  display  in  question;  mark  E if  either  hand  could  be  used!  Y ^ 
Please  make  use  of  the  following  additional  information  to  guide  your  answers. 

Context  #1:  You  are  flying  contour  in  an  AH-64  Apache  ( version  ? ) helicopter  You  am  vfr  in 

2S££Y5S tfpT  ta0B????  Y“  » al“  " •» 


Context  #2:  COMM 


Context  #3:  Emergency  procedure  (?) 
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SAMPLE  PROXIMITY  MATRIX 
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N = None  L = Low  M = Medium  H = High  V = V«yhigh 


SAMPLE  proximity 
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FUNCTIONAL  PROXIMITY  MATRIX 


orque  x»tor  air  attitude  radar  /ert.  accd-  ;tandb\  com-  clock 
ipeed  >peed  aid-  ;peed  oration  :ompa™s 

neter 


torque 


acceleration 


standby 

compass 


compass 


clock  *xxx  XXXX  *XXX  *xxx  *XXX  xxxx  xxxx  xxxx  xxxx  xxxx 

*XXX  KXXX  *xxx  *XXX  XXXX  XXXX  XXXX XXXX  XXXX  KXXX  XXXX 

N = None  L = Low  M = Medium  H = High  V = Very  high 

EES"  proximit>  ?s  ba^ed  on  an  estimate  of  the  extent  to  which  two  indicators  must  be 

§! 

<H§»  if  they  t o to  bS  « al^d^h^  “>ge,her  “d M (McdIm> OT  H 
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PHYSICAL  PROXIMITY  MATRIX 
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CORRELATIONAL  PROXIMITY  MATRIX 


torque 

■otor 

speed 

air 

speed 

ittitude 

radar 

titi- 

neter 

v'ert. 

speed 

iccel- 
: ration 

standb 

;ompa 

> com- 
s>pass 

clock 

torque 

22 

■ 

rotor  speed 

■ 

jg 

■ 

■ 

■ 

airspeed 

g 

\wM 

attitude 

22 

22 

22 

g 

1 

radar  altimeter 

22 

g 

22 

g 

xxxx 

■ 

vertical 

airspeed 

■ 

■ 

■ 

■ 

acceleration 

22 

i 

g 

g 

g 

■ 

■ 

■ 

standby 

compass 

HI 

jjjj 

■ 

■ 

1 

■ 

compass  } 

_ 1 

jggj 

8! 

g; 

g 

■ 

gj 

gj 

■ 

22 

■ 

■ 

clock  ? 

(XXX  > 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX 

? 

(xxx? 

(xxx? 

(XXX  ? 

(XXX  ? 

(XXX  ? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX? 

(XXX 

N - None  L = Low  M = Medium  H = High  V = Very  high 

™,e'a‘j0nal  Prfoximity  is,the  deg.ree  10  which  the  values  shown  by  two  indicators  are 
correlated  over  a four  second  time  window.  For  example,  airspeed  and  power  would  be  hiehlv 
correlated  but  bank  and  altitude  might  have  low  correlation  of  values;  that  is,  bank  might  be  jus^  as 

likely  to  be  level  as  banked,  in  low  and  high  flight.  8“  oc  just  as 
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location  of  control  action 


torque 

rotor  speed 

airspeed 

attitude 

radar  altimeter 

vertical 

airspeed 

; I 

acceleration 

standby 

compass 

compass 

clock 

Location  of  control  action  associated  with  the  display: 

NA  = Not  Applicable  L = Left  hand  R = Right  hand  E = Either  hand 


Left: 

Right: 

Either 


ontrol  action  is  normally  performed  with  the  left  hand 
ontrol  action  is  normally  performed  with  thenght  ha 
ontrol  action  can  be  performed  with  either  hand 
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Annex  G 


Army-NASA  Aircrew/Aircraft  Integration  Program:  Phase  IV 


a3h 


Man-Machine  Integration  Design  and  Analysis  System  (MIDAS) 
Software  Detailed  Design  Document 


Anthropometric  Model  (JACK) 


prepared  by 
Christian  Neukom 
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Figure  1 . Jack/A3I  Workstation  Interaction . 

Figure  2.  A graph  of  a peabody  environment  showing  the  CMnectivity  of  objects.”  '..!  5 


u 


aaam  machinF  INTEGRATION  DESIGN  & ANALYSIS  SYSTEM 
(MIDAS)  SOFTWARE  DETAILED  DESIGN  DOCUMENT 
' PHASE  IV: 


ANTHROPOMETRIC  MODEL  (JACK) 


1.0  INTRODUCTION 


1.1  IDENTIFICATION  OF  DOCUMENT 

This  is  the  software  Product  Specification  for  the  Anthropometric  Model  (Jack ) component 
If  he  MTOAS  Software  System.  Description  of  the  detailed  processing  requirements, 

provided  4 each  lower  levelComputer  Software 
Component  (CSC),  unit,  or  function  contained  within  this  module. 

1 2 SCOPE  OF  DOCUMENT 

gSpWcs  and  should  be  familiar  with  a high  level  computer  language. 

1.3  PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 

A fundamental  requirement  of  MIDAS  is  a physical  represe tf  the 

develo^d  ^cmgh  B^leraf^Universityof  Pennsylv^ma.Jack  *S 

ssfesassss  %£s?£ze£  * 

fit,  and  visibility. 

2.0  RELATED  DOCUMENTATION 

2.1  APPLICABLE  DOCUMENTS 

The  following  documents  are  referenced  herein  and  are  directly  applicable  to  this  volume. 

Parrv  R phillins  Jack  User's  Guide,  Jack  Version  4,  Computer  graphics  Laboratory, 

Department  of  Computer  and  Inf°™a^n  ^ciXC20  "l989 of  PennSy  Vama’ 
Philadelphia,  Pennsylvania  19104-6389,  October  20, 1989. 


Page  G- 1 


Cany  B.  Phillips,  Programming  with  Jack,  Computer  Graphics  Laboratory  Department  of 

Vit°ctaP1987  'InC''  "!'S'4D  USm  GMe'  V°'"me ' and  V'rei»"  11.  Mountain 

2.2  INFORMATION  DOCUMENTS 

The  following  documents  amplify  or  clarify  the  information  presented  in  this  volume: 

fSSSSSSsa® 

3.0  CONCEPT 

3.1  USER  DEFINITION 

The  Jack  module  is  used  by  A3I  cither  interactively  by  a human  operator  or  controlled 
ough  an  integrated  simulation  as  demonstrated  during  our  phase  IV  demo  In  the 

2SSE  TS' JaCk  3CCepJS  COmmands  via  menu  selection,  keytoard  en^y  and  animation 
Model  • tk  inte8ratecl  mode,  Jack  receives  instructions  from  the  Symbolic  Operator 
Model  via  the  Cotnmumcanon  manager.  Figure  1 displays  the  various  inSKfX 

Jack^software  w„h  the  A*  computers,  flies,  and  system  programs,  as  well  asTe  human 
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Figure  1.  Jack/A^I  Workstation  Interaction. 

4.0  requirements 

4.1  HARDWARE  ENVIRONMENT 
The  Jack  software  runs  on jvmousIRIS 

SSS  ffils  4MoSt  or  better  is  r£,uired  ,o  run  all  the  available  functions  and 
to  obtain  a satisfactory  response  time. 

The  machines  available  a.  A>,  are  die  IRIS  4D.2™  the  IRIS  4D220GTXB.  The 
technical  specification  for  these  machines  are  listed  below . 


W-4D120GTX  Power  Series  Workstation  (Coral)  consisting  of. 

• 32MB  CPU  memory 

• 380MB  Winchester  disk  drive 
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• SCSI  1/2"  tape  drive,  built  in 

• image  memory 

bi,planB  <8  biB  fOT  «ch  «*  «~. 
24  for  double  buffcred  alpha 

4 1280  * 1024  overlay  or  underlay  bit  planes 
4 window  ID  bit-planes  (total  96  bits/pixel) 

• Ethernet  interface  card 

• 19"  high  resolution  monitor 

• keyboard  & mouse 


W-4D220GTXB  Power  Series  Workstation  (Starfish)  consisting  of: 


• 32MB  ECC  CPU  memory 

• 380MB,  and  780MB  ESDI  Winchester  disk  drives 

• SCSI  1/2"  tape  drive,  built  in 

• image  memory 


i 


blue;  double-buffered. 

1$  J28?  * ^24™:'8e  bit-planes  for  double  buffered  alpha 
24  bit  Z-buffer  (1280  * 1024) 

4 1280  * 1024  overlay  or  underlay  bit  planes 
4 window  ID  bit-planes  (total  96  bits/pixel) 

• Ethernet  interface  card 


• 19"  high  resolution  monitor 

• keyboard  & mouse 


4.2  SOFTWARE  ENVIRONMENT 

• Operating  system:  IRIX  V system  release  4D1-3  2 

• Network  File  System  Software 

• C programming  language 

• C++  translator,  release  1.0  source  code 

• IRIS  graphics  library 

• 4 sight  window  manager 

5.0  DESIGN 


5.1  ARCHITECTURAL  DESIGN 

egment  in  the  environment  is  represented  by  a polyhedron  surface  (psurf)  which  is  a 
boundary  representation  using  tables  of  nodes,  edges,  and  faces.  ' 

?CfHita  f°[  disp!aying  and  manipulating  objects  represented  by  peabody  It 
provides  a standard  user  and  programmer  interface  to  routines  which  operate  onthe 
~ent;HJack  b/  if  primarily  a user  interface,  and  its  princi^  Son  ?s 
maintaining  the  windows  and  control  input  from  the  user  in  a consisted  way 
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y 


A 


world 

coordinate  system 


Fisure  2.  A graph  of  a peabod,  environment  showing  the  connectivity  of 
b objects. 

root  site  roughly  corresponds  to  the  origin  oft  e g , ad  P specified  relative  to 

£ That' means  that  each 

psurf  is  designed  in  its  own  coordinate  system. 

6.0  USER'S  GUIDE 

&gegsBBEB&E£; 

a2«ssasa  apes 

»thc  descri^ioi^indre  u ser'  s^ian  ual  and  can 
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on  r£lanLa  TiCk  T^nCu  ^ppendix  C shows  the  ^ck  directory  structure  as  it  appears 
the  tape.  Appendix  E,  the  Readme  File,  gives  instructions  for  installing  Jack  from  tape. 

6.1  DESCRIPTION  OF  JACK  FILES  CREATED  IN  PHASE  IV 

6.1.1  Psurf  Files  (.pss) 


lb_fwd01a.pss 


lb_fwd!3.pss 


This  collection  of  22  files  make  up  the  copilot- 
gunner 

cockpit  of  a Longbow  helicopter.  Included  in 
these  files 

is  part  of  the  fuselage  surrounding  the  cockpit. 


Ib_aft01.pss 


lb_aftl  3. pss 


These  2 1 files  make  up  the  cockpit  of  the  pilot  of  a 
Longbow  helicopter.  Included  in  these  files  is 
part  of 

the  fuselage  sunrounding  the  cockpit. 


Ib_ext01a.pss 


lb_extlOc.pss 


These  21  files  make  up  the  exterior  of  a Longbow 
helicopter. 


ah64_hta.pss 


ah64_htd.pss 

ht_visor.pss 

lfviewcone.pss 

lviewcone.pss 

rfviewcone.pss 

rviewcone.pss 

sfrviewcone.pss 

slviewcone.pss 

srviewcone.pss 

ssphere.pss 


These  5 files  collectively  make  up  an  Apache 
IHADDS 

helmet  includi  ^ visor  and  monacle. 

Static  viewcone  for  vision  analysis. 

Field  of  view  (FOV)  view  cones  for  vision 

analysis. 


This  file  is  used  in  the  vision  demo. 


6.1.2 


Environment  Files  (.env) 
05pilot.env 
bio_pilot_he!met.env 


Fifth  percentile  polybody  figure  in  pilot  posture. 

Biostereometric  figure  with  IHADDS  helmet  in 
pilot  posture. 
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bio_pilot_helmet_lb_aft.env 


Biostereometric  figure  with  IHADDS  helmet  in 
pilot  posture  and  in  longbow  pilot  cockpit 
environment. 


bio_pilot_lb_aft.env 

d3.env 

d5.env 

d6.env 

demol.env 

demo2.env 

demo3.env 

fwd_bio_pilot.env 

helmet80.env 

lb_aft_bio_pilot.env 

lb_aft_pilotl.env 

lb_ext_lbio_pilot.env 

lb_ext_2bio_pilot.env 

lb_ext_2pilot.env 

lb_f  wd_bio_f  ilot  .env 

lb_fwd_pilot.env 


Biostereometric  figure  in  pilot  posture  and  in 
Longbow  pilot  cockpit  environment. 

Longbow  copilot-gunner  cockpit  with  polybody 
figure  seated  in  crew-position.  No  fuselage  and 
back  of  seat. 

Longbow  helicopter  with  two  biostereometric 
figures  seated  in  crew  position. 

Two  polybody  figures.  One  carrying  box  (reach 
constraints  between  hands  and  box). 

Line-up  of  male  and  female  polybody  figures  (5th, 
50th,  95th  percentile)  and  95th  percentile  by 
height  biostereometric  male  figure. 

Biostereometric  figure  wearing  IHADDS  helmet. 

Biostereometric  figure  wearing  helmet  seated  in 
copilot-gunner  cockpit. 

Biostereometric  figure  in  pilot  posture. 

IHADDS  helmet.  Requires  ah64_htx.pss  and 
ht_visor.pss  files. 

Longbow  pilot  cockpit  with  biostereometric  figure 
seated  in  crew  position. 

Longbow  pilot  cockpit  with  polybody  figure 
seated  in  crew  position. 

Longbow  helicopter  with  biostereometric  figure 
seated  in  copilot  cockpit. 

Longbow  helicopter  with  two  biostereometric 
figures  seated  in  crew  position. 

Longbow  helicopter  with  two  polybody  figures 
seated  in  crew  position. 

Longbow  CPG  cockpit  with  biostereometric 
figure  wearing  IHADDS  helmet. 

Longbow  CPG  cockpit  with  95th  percentile 
polybody  figure. 
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Ib_fwd_pilot05_helmet.env  Longbow  CPG  cockpit  with  5th  percentile 

polybody  figure  wearing  helmet. 

Longbow  CPG  cockpit  with 

Longbow  CPG  cockpit  with  95th  percentile 
biostereometric  figure  and  IHADDS  helmet. 

Longbow  pilot  cockpit  with  95th  percentile 
polybody  figure  seated  in  crew  position. 

pilot05_helmet  JbJwd.env  5th  percentile  polybody  figure  wearing  IHADDS 

helmet  in  seating  posture  in  copilot  cockpit 

environment. 


p31.env 

p32.env 

p33a.env 


pilot05_lb_.fwd.env 


pilot_helmet.env 


pilot_]b_aft.env 


pilot_lb_fwd.env 


95th  Percentile  polybody  figure  in  seated  posture 
in  copilot  environment  (Can  be  imported  into 
copilot  cockpit). 

5^  percentile  polybody  figure  in  seating  posture 
in  copilot  cockpit  environment. 

95'h  percentile  polybody  figure  in  seated  posture 
in  longbow  pilot  environment  (Can  be  imported 
into  longbow  pilot  cockpit). 

9Slh  Percentile  polybody  figure  in  seated  posture 
in  copilot  environment  (Can  be  imported  into 
longbow  copilot-gunner  cockpit). 


6.1.3  Jack  Command  Language  Files  (.jcl) 


dl.jcl 

d2.jcl 


d3.jcl 


d5.jcl 


Loads  demol.env  file  and  makes  it  shaded. 

Loads  demo2.env  file.  Turns  projections  off, 
displays  4-panel  screen,  makes  window  shaded, 
and  makes  visor  transparent. 

Loads  d3.env,  and  sets  up  animation.  In  the 
animation,  the  copilot-gunner  seated  in  the  cockpit 
reaches  for  the  left  MFD  with  his  right  hand.  This 
animation  also  requires  the  files  d3.scr  and  d3.act. 

Loads  d5.env,  turns  on  6 lights,  and  makes  scene 

shaded. 


d6.jcl 


Loads  demob,  env,  makes  it  shaded,  and  sets  up 
interactive  balance  reach.  Creates  reach  constraint 
between  figure  and  box  for  constraint  demo. 
Requires  box.env  file. 
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p31.jcl 

p32.jcl 

p33a.jcl 

vis.jcl 


Setup  for  p31.env  (shading,  transparency,  etc.). 
Setup  file  for  p32.jcl 
Setup  file  for  p33a.env 
Setup  file  for  vision  demo. 


6.1.4  Action  Files  (.act) 

d3.act 


This  file  is  used  by  d3.jcl.  It  contains  the  action 
script. 


6.1.5  Script  Files  (.scr) 

d3.scr 


This  file  is  used  by  d3.jcl. 


6.1.6  Data  Files 

QO_il_dl.cons 

QO_il_H_dl.dat 

QO_i2_ll_dl.cons 

QO_i2Jl_dl.dat 

QO_i3_ll_dl.cons 

QOJ3_ll_dl.dat 

QOJ4Jl.5_dl.cons 

QOJ4_11.5_dl.dat 

QO  J4  _1 1 _d  1 .cons 

Q0J4_ll_dl.dat 

QOJ4_12.54_dl.cons 

QO_i4_12.54_dl.dat 

QO  _i4  _12_dl  .cons 

QO_i4_12_dl.dat 

cone_den.left 

cone_den.right 

contour.left 

contour.right 

frviewcone 

lviewcone 

qo_iO_ll_dl.cons 

qojl  _ll_dl.cons 

qo_i2_ll_dl.cons 

qo_i3_ll_dl.cons 

qo_i4_ll.5_dl.cons 

qo_i4_ll_dl.cons 

qo_i4_12.54_dl.cons 


Vision  confusion  data  and  their  contour  data  for 

large 

font  size. 


Cone  density  data  and  contours. 

Iso  contour  data. 

Static  viewcones. 

Confusion  data  and  contours  for  small  font  size. 
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qo_i4_12 d 1 .cons 

rfviewcone 


Field  of  view  (FOV)  viewcones 


6.2  DOWNLOADING  PSURF  FILES  FROM  CDE 

Simply  Sw  5iVS^?rthVhlbCen- l0aded  inL°  theF?E’ il  can  then  1x5  downloaded  to  Jack. 
.hTrnc  h CJ  ^ h environment  that  will  be  downloaded  on  Jack.  Then  go  to 
e CDE  menu  and  choose  Data  Format".  A window  will  appear  at  the  bottom  that  asks 
for  one  of  several  types  of  file  formats.  Select  the  Psurf  format  and  click  on  OK.  This 
away  and  another  window  will  appear  in  the  center  of  the  screen  that  will 

Sled  ,o  fdSm  Se  “ rcmn  Shma  be  lyped  U""ss  **  *"’>**’  is  10  •» 

This  window  will  be  replaced  by  another  window  asking  for  a filename  to  be  typed  in 
This  is  the  filename  that  the  psurf  will  be  saved  in.  This  window  will  go  awayS another 
will  appear  in  the  upper  left  comer  that  has  the  legend  ''Select  Bead  Then  Hit  OKor  Done" 
and  three  buttons. ..OK,  Done,  and  Undo.  Now,  to  choose  a section  of  the  environment 

fdtyTen  c^  °°  that  fleflion  so  ,that  a white  ljne  appears  around  the 

eage,  then  click  on  OK  . Each  section  will  be  saved  in  the  order  it  is  selected  When  all 

Ihe  secdons  have  been  selected,  clicking  on  Dune  ' save  all  those 

Jack  keens  the  co“ins.color  information  which  Jack  is  not  able  to  read  directly  . 

r eeps,the  color  ^formation  in  an  environment  file.  To  fix  this  problem,  a short  awk 
p oeram  was  written  to  extract  the  color  infomiation  from  the  psurf  files  and  to  write  them 

,he  same  nam- »«•“  - *>  - teisss  sr 

tiv» k -f  pss.awk  psurt.pss  > psuif.cnv 

7.0  ABBREVIATIONS  AND  ACRONYMS 


A?I 

CDE 

CSC 

CSCI 

Psurf 

UPenn 

SASS 

8.0  GLOSSARY 


Army-NASA  Aircrew  Aircraft  Integration 

Cockpit  Design  Editoi 

Computer  Software  Component 

Computer  Software  Configuration  Item 

Polyhedron  surface 

University  of  Pennsylvania 

Spreadsheet  Anthropometric  Scaling  System 


9.0  NOTES 

9.1  LIMITATIONS 

Jack  is  limited  by  the  graphics  and  computation  power  of  the  machine  it  runs  on 

slowdown 1 thC  highl>' dCtailed  1 °nfib0W cockPi,s or biostereometric  figures 
, ,,  ,.  f.  8rdPb]cs  processing  enormously,  sometimes  to  a point,  where  the  operator 

looses  the  feeling  of  interactive  control.  We  have  worked  around  this  problem  by 
simp  Tying  the  environments  we  work  with  to  a bare  minimum.  However,  to  take  full 
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advantage  of  our  detailed  environments  and  all  the  capabilities  of  Jack,  a high  end  model 
Iris  workstation,  such  as  the  IRIS-4D-220GTX,  is  required. 

9.2  FUTURE  DIRECTIONS 

figure  can  be  represented  in  Jack. 

„ Will  also  be  necessary  » 

the^mrneThenDTacemSt  of  the  figu/affects  the  reaclf analysis  and  the  view  from  the 

bS  conSions  and  functions,  such  as  the  placement  of  the  mannequin  into  the  cockpit  and 
the  reach  analysis. 


10.0  APPENDICES 
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APPENDIX  A — MENU  ROAD  MAP 
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Menu  Roadmap 


Main  Menu 


main  menu 

view  menu 

=> 

create  menu 

=> 

write  menu 

=> 

edit  menu 

=> 

info  menu 

=> 

option  menu 

animation  menu 

=> 

quit 

View  Menu 


main  menu 

• 

view  menu 

=> 

• 

=> 


view  menu 


change  view 
snap  view  to  site 
move  camera 
move  reference 
set  field  of  view 
attach  view  to  site 
reset  view  to  camera 
set  horizontal  view  scale 
set  vertical  view  scale 
set  zoom  view  scale 
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Create  Menu 


main  menu 

• 

create  menu 

=> 

* 

=> 


create  menu 


read  file 


read  environment  file 


read  figure  file 


read  peabody  string 


create  figure  from  psurf 
create  site 


create  constraint 


create  light 


body  menu 


=> 


primitive  menu 


Body  Menu 


main  menu 

• 

create  menu 

=> 

; 

=> 


create  menu 


body  menu 


=> 


body  menu 

create  50th  %ile  male 

create  50th  %ile  female 

create  5th  %ile  male 

create  5th  %ile  female 

create  95th  %ile  male 

create  95th  %ile  female 

create  50th  %ile  polybody  male 

create  50th  %ile  polybody  female 

create  5th  %ile  poiybody  male 

create  5th  %ile  poly  body  female 

create  95th  %ile  polybody  male 

create  95th  %ile  polybody  female 

create  95th  %i!e  male  contour  body 
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Primitive  Menu 


main  menu 

■ 

create  menu 

=> 

; 

=> 


j primitive  menu 

create  cube 
create  small  cube 
create  cylinder 
create  pyramid 
create  torus 
create  ground  plane 
create  ground  mesh 


Write  Menu 


main  menu 

* 

write  menu 

=> 

, 

write  menu 


write  environment 

write  positions 

write  figure  definition 

write  segment  as  psurf 

write  segment  as  global  psurf 

write  figure  psurfs 

write  JCL  log 


Edit  Menu 


main  menu 

• 

edit  menu 

=> 

. 

=> 


edit  menu 

move  menu 

==> 

object  menu 

=> 

reach  menu 

=> 

constraint  menu 

=> 

attribute  menu 

=> 

light  menu 

=5> 

CSG  menu 

=> 
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Reach  Menu 


main  menu 

| 

• 

=> 

=> 

_5>J 

; 

reach  menu 


interactive  reach 
reach  site 
multiple  reach 
turn  active  on 
turn  active  off 
turn  steps  on 
turn  steps  off 


Constraint  Menu 


main  menu 

i 

edit  menu 

! 

constraint  menu 

create  reach  constraint 

■ 

• 

=> 

delete  reach  constraint 

edit  menu 

=> 

=> 

constraint  menu  => 

delete  all  reach  constraints 

■ 

; 

change  reach  constraint 

set  reach  iteration  time  limit 

Attribute  Menu 


main  menu 

edit  menu 

1 

1 : 

• 

edit  menu 

=> 

=> 

attribute  menu  => 

• 

• 

attribute  menu 

set  attribute  diffuse 

set  attribute  ambient 

set  attribute  specular 

set  attribute  glossiness 

make  face  smooth 

make  face  flat 

make  make  segment  smooth 

make  segment  flat 

give  face  new  attribute 

give  face  old  attribute 

give  segment  new  attribute 

give  segment  old  attribute 

give  csurf  new  attribute 

give  csurf  old  attribute 
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Option  Menu 


option  menu 

window  menu 

=> 

background  menu 

=> 

display  menu 

=> 

color  menu 

=> 

parameter  menu 

=> 

simulation  menu 

=> 

retina  display 

=> 

retina  editor 

=> 

vision  information 

=> 

trace  menu 

=> 

command  menu 

=> 

Window  meu 


main  menu 


• 

option  menu 

=> 

• 

=> 


window  menu 

ortho  window  menu  ==> 

make  window  shaded 

make  window  wireframe 

make  window  ordinary 

create  shaded  window 

create  wireframe  window 

create  ordinary  window 

delete  window 

freeze  window 

thaw  window 

turn  camera  on 

turn  camera  off 

set  window  location 

make  viewer  local 

make  viewer  infinite 

set  window  ambient 

set  window  attenuation  ! 

write  window  image 

option  menu 

• 

window  menu 

=> 

; 
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Ortho  window  menu 


option  menu 


=> 


window  menu 


=> 


window  menu 

ortho  window  menu 

• 

four  panel  screen 

ortho  window  menu 

=5> 

create  x window 

• 

create  y window 

create  z window 

Background  Menu 


f main  menu 

. 

1 #j  2 jjjjy^  ^ 

• 

=> 

• 

background  menu 

turn  background  off 

turn  background  on 

turn  grid  off 

turn  grid  on 

turn  stars  off 

turn  stars  on 

make  background  shaded 

make  background  wireframe 

set  grid  blocks 

turn  projections  on 

turn  projections  off 

turn  x projections  on 

tum  x projections  off 

turn  y projections  on 

tum  y projections  off 

turn  z projections  on 

turn  z projections  off 
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Display  Menu 


main  menu 

! 

n mam 

• 

; 

=> 

=> 

display  menu  => 

pm 

; 

=> 


make  everything  shaded 

make  everything  wireframe 

make  figure  shaded 

make  figure  wireframe 

make  segment  shaded 

make  segment  wireframe 

make  segment  transparent 

turn  segment  on 

turn  segment  off 

turn  site  on 

turn  site  off 

turn  segment  sites  on 

turn  segment  sites  off 

turn  nodes  on 

turn  nodes  off 

turn  edges  on 

turn  edges  off 

face  enumeration  on 

face  enumeration  off 

turn  segment  projections  on 

turn  segment  projections  off 

turn  figure  projections  on 

turn  figure  projections  off 
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Color  Menu 


Parameter  Menu 


parameter  menu 

set  scene  scale 
set  scene  aux  scale 
set  line  width 
set  distance  units 
set  distance  precision 
set  angle  units 
set  angle  precision 
set  mass  units 
set  mass  precision 
set  rotation  type 
set  peabody  path 
set  view  glide 
set  move  glide 
change  directory 
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Simulation  Menu 


555 

| 

• 

• 

=> 

=> 

simulation  menu 

■■■ 

• 

=> 


simulation  menu 


simulation  off 


set  simulation  level  1 


set  simulation  level  2 


single  set  simulation 


read  squash 


generate  scaling  squash 


bind  joint  to  port 


Retina  Display 


main  menu 

1 

• 

=> 

=^> 

; 

=> 


retina  display 


create  retina  window 

create  fiels  of  view  cones 

fixate  eyes  on  site 

interactive  fixation 

monocular/binocular  map 
zoom  in/out  of  retina  window 
show  eye  scan  
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Retina  Editor 


option  menu 

=> 

• 

1 retina  editor 

create  retina  editor 

draw  left  retina  object 

draw  right  retina  object 

draw  filled  retina  objects 

retroject  left  retina 

retroject  right  retina 

retroject  both  retina 

clear  both  retina 

clear  retina  objects 

save  retinda  objects 

load  retina  objects 

load  QO  confusion  data 

load  qo  confusion  data 

load  Iso  Focus  contour  data 

load  cone  density  data 

zoom  In/Out  of  editor  window 

Vision  Information 


main  menu 

option  menu 

vision  information 

create  adaption  info  window 

=> 

vision  information 

=> 

create  legend  window 

• 

create  Aitoff  window 

• 

initialize  state  info 

i 

set  state  information 
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Trace  Menu 


main  menu 


=> 


1 

! 

bsd^hi 

B 

IS3XSHE9 

=> 

trace  segment 

r ; 

untrace  segment 

set  trace  color 

clear  trace 

delete  trace 

Command  Menu 


Eo^S 

! 

• 

kssmsm 

=> 

i 

; 

=> 


| command  menu 

help  for  command 

help  by  subject 

keymode  on 

keymode  off 

disable  WM  quit 

disable  graphics 

enable  graphics 

bind  command  to  key 

describe  key  bindings 

create  communication  port 

send  to  communication  port 

close  communication  port 
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Animation  menu 


animation  menu  j 

help 

describe  script 

group  menu  =i^ 

action  menu 

playback  menu 

read/write  record  menu 

initialize  animation 

create  keyframe  from  current  state 

delete  key  frame 

change  keyframe  to  current  state 

change  keyframe  normal  time 

change  frame  rate  normal  time 

erase  animation 

Group  Menu 


main  menu 

animation  menu 

• 

animation  menu  =^> 

=> 

1 group  menu 

* 

: 

• 

=> 


group  menu 

create  figure  groups 
create  figure  joints  group 
create  figure  location  group 
create  camera  group 
create  joint  group 

| create  segment  nodes  group 
create  mixed  group 
change  group  name 
describe  group 
delete  group 
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Action  Menu 


EEBSSHH 

• 

animation  menu 

=> 

■ 

• 

=> 


play  back  menu 


choose  current  action 

create  action 

change  action  length 
change  action  start  time 
change  action  interpolation 
change  action  name 
describe  action 
delete  action 


Playback  Menu 


main  menu 

• 

animation  menu 

=> 

play  back  menu 

; 

• 

=> 


play  back  menu 

read  script  file 

read  action  file 

step  throu  interpolated  frames 
repeat  playback 
playback  in  rea  time 
special  playback 
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Read/Write  Record  Menu 


main  menu 


; 

— 

animation  menu 

=> 

• 

=> 


animation  menu 


read/Write  record  menu 

=> 

• 

read/write  record  menu 

read  script  file 
read  action  file 
write  script  file 

write  action  file  — — — — ~™ 

save  complete  environment  file  for  rendering 
[ save  incremental  environment  files  for  rendering 
record  animation 
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APPENDIX  B 


— DESCRIPTION  OF  MENU  SELECTABLE  COMMANDS 
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Description  of  Menu  Selectable  Commands 


VIEW  MENU 

change  view 


snap  view  to  site 
move  camera 
move  reference 
set  field  of  view 

attach  view  to  site 
reset  view  to  camera 

set  horizontal  view  scale 
set  vertical  view  scale 
set  zoom  view  scale 


Interactively  changes  view  angle,  direction,  and  zoom.  The  left 
mouse  button  controls  the  horizontal  swing,  the  middle  mouse 
button  the  vertical  swing,  and  the  right  mouse  button  the  zoom. 
With  the  control  key  pressed,  change  view  executes  the  same 
function  but  in  the  "panning"  mode.  ESC  terminates  the  process 
or  alternatively,  AC  terminates  and  resets  the  view  to  its  original 
position. 

Changes  the  view  reference  point  to  a new  location. 

Changes  view  "non-interactively" 

Changes  the  view  reference  point 

The  default  field  of  view  is  40°.  Angles  of  more  than  70°  give 
distorted  images. 

Moves  view  to  a specified  site. 

Resets  view  after  it  has  been  changed  by  some  of  the  above 
commands. 

Changes  the  sensitivity  of  the  left  mouse  button  for  viewing. 
Changes  the  sensitivity  of  the  middle  mouse  button  for  viewing. 
Changes  the  sensitivity  of  the  right  mouse  button  for  viewing. 


CREATE  MENU 

read  file 

read  environment  file 
read  figure  file 
read  peabody  string 
create  figure  from  psurf 
create  site 
create  constraint 


Files  with  the  following  extensions  can  be  read  with  this 
command:  .env,  .fig,  .pss,  .bps,  .jcl. 

Reads  an  environment  file  if  the  suffix  is  not  .env. 

Reads  a figure  file  if  the  suffix  is  not  .fig. 

Reads  a string  of  peabody  syntax. 

Reads  a psurf  file  if  suffix  is  not  .pss. 

Prompts  for  segment  on  which  to  create  site. 

Do  not  confuse  this  with  the  reach  constraints. 
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create  light 


Creates  a light  source. 


BODY  MENU  t 

(The  following  commands  create  the  specified  figures.) 

create  50th  percentile  male 

create  50th  percentile  female 

create  5th  percentile  male 

create  5th  percentile  female 

create  95th  percentile  male 

create  95th  percentile  female 

create  50th  percentile  polybody  male 

create  50th  percentile  polybody  female 

create  5th  percentile  polybody  male 

create  5th  percentile  polybody  female 

create  95th  percentile  polybody  male 

create  95th  percentile  polybody  female 

create  95th  percentile  by  height 
biostereometric  male 

PRIMITIVE  MENU  J u.  x 

(The  following  commands  create  the  specified  objects.) 

create  cube 
create  sphere 
create  cylinder 
create  pyramid 
create  torus 
create  ground  plane 
create  ground  mesh 

WRITE  MENU 
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write  environment 


write  positions 
write  figure  definition 
write  segment  as  psurf 
write  segment  as  global  psurf 
write  figure  psurfs 
write  JCL  log 

MOVE  MENU 

move  figure 
adjust  joint 

move  figure  with  view 

reset  joint 
reset  figure 
move  site 
move  node 

move  edge 
move  face 

OBJECT  MENU 

delete  environment 
delete  window 

delete  constraint 


Writes  the  entire  environment  to  a file  which  should  have  the 
suffix  .env. 

Writes  the  positions  of  all  the  figures  in  the  environment. 
Writes  figure  file  of  a specifies  figure. 

Writes  segment  psurf  to  a file  as  is. 

Writes  segment  as  global  psurf. 

Writes  figure  as  a global  psurf. 

Writes  script  file  of  interactive  session. 


Move  figure  to  new  position. 

Joint  angles  may  be  adjusted,  one  degree  at  a time. 

Attaches  the  view  to  a figure  and  allows  moving  the  view. 
When  done,  it  reattaches  the  view  to  die  the  previous  site. 

Resets  joint  to  its  default  position. 

Resets  figure  to  its  default  position. 

Move  a site  to  a new  location 

Moves  node.  This  command  alters  the  coordinates  of  the  psurf 
node  but  it  does  not  alter  the  file  from  which  it  was  originally 
read.  To  save  the  changed  geometry,  write  the  psurf  back  to 
file. 

Moves  a pair  of  nodes.  Same  rules  as  for  save  node. 

Moves  all  nodes  in  a selected  face. 


Deletes  the  environment  in  the  current  window. 

Deletes  a selected  figure  from  an  environment.  If  the  figure  has 
a constraint  attached  to  it,  it  will  be  deleted  too. 

Removes  a constraint. 
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reroot  figure 

scale  figure 
set  joint  type 
freeze  joint 

Thaw  joint 
rename  figure 
rename  segment 
rename  site 
rename  joint 


This  command  changes  the  root  site  of  a figure.  In  the  process, 
the  figure  "location"  is  also  changed  since  the  location  is  the 
global  placement  of  its  root  site. 

Each  psurf  is  scaled  in  its  own  coordinate  system. 

Sets  the  degree  of  freedom  in  a joint. 

A frozen  joint  may  not  be  manipulated  and  will  not  be  changed 
by  the  inverse  kinematics  algorithm. 

Unfreezes  a joint. 

Prompts  for  new  figure  name. 

Prompts  for  new  segment  name. 

Prompts  for  new  site  name. 

Prompts  for  new  joint  name. 


REACH  MENU 

interactive  reach 

reach  site 

multiple  reach 

turn  active  reach  on 

turn  active  off 
turn  steps  on 

turn  steps  off 


Allows  you  to  interactively  drag  an  end  effector.  Prompts  for 
goal  type,  end  effector  site,  and  a base  joint. 

Allows  to  specify  single  reach  goal.  Prompts  for  goal  type,  goal 
site,  end  effector,  and  base  joint. 

Allows  to  specify  several  reach  goals.  Prompts  for  goal  types, 
goal  sites,  end  effectors,  base  joints,  and  weight  factor. 

Adds  more  active  segments  to  reach  chain  if  goal  cannot  be 
reached  with  the  specified  joint  chain. 

Default  setting.  Only  specified  joint  chain  is  used  for  reaches. 

Default  setting.  Intermediate  position  of  the  figure  are  displayed 
as  reach  algorithm  is  executed. 

Intermediate  positions  are  not  displayed  as  algorithm  is 
executed. 


CONSTRAINT  MENU 
create  reach  constraint 


delete  reach  constraint 


A restriction  to  the  reach  commands  is  created.  Prompts  for 
goal  type,  goal  site,  end  effector,  base  joint,  and  weight  factor. 

Removes  constraint  imposed  by  create  constraint  . 


delete  all  reach  constraints  Removes  all  constraints. 
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change  reach  constraint 
set  reach  iteration  time  limit 


Uses  default  parameters  of  the  previous  reach  constraint 
Puts  a time  limit  on  each  iteration  step  for  the  reach  algorithm. 


ATTRIBUTE  MENU 

set  attribute  diffuse 

Sets  surface  attribute  "diffuse"  of  an  object  Describes  the  color 
of  an  object  when  it  is  illuminated. 

set  attribute  ambient 

Sets  surface  attribute  "ambient"  of  an  object.  Describes  the  color 
or  an  object  when  it  is  not  illuminated. 

set  attribute  specular 

Sets  surface  attribute  "specular"  of  an  object  Describes  the 
color  of  the  specular  highlights  of  an  object. 

set  attribute  glossiness 

Sets  surface  attribute  "glossiness”  of  an  object  Describes  the 
specular  scattering  of  the  surface. 

make  face  smooth 

Applies  Phong  shading  to  specified  face. 

make  face  flat 

Default.  The  surface  of  an  object  is  modeled  as  discrete 
polyhedron. 

make  segment  smooth 

Applies  Phong  shading  to  specified  segment 

make  segment  flat 

No  Phong  shading. 

give  face  new  attribute 

Creates  a new  attribute  and  associates  it  with  the  specified  face. 

give  face  old  attribute 

Allows  to  select  a previously  defined  attribute  and  associate  with 
specified  face. 

give  segment  new  attribute 

Creates  a new  attribute  and  associates  it  with  the  specified 
segment.  r 

give  segment  old  attribute 

Allows  to  select  a previously  defined  attribute  and  associate  with 
specified  segment. 

give  csurf  new  attribute 

Sets  the  attribute  association  for  all  faces  which  are  connected  to 
a selected  face. 

give  csurf  old  attribute 

Returns  old  attribute  to  csurf. 
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LIGHT  MENU 

set  light  color 

set  light  ambient 
make  lights  local 

make  lights  infinite 

make  lights  visible 

make  lights  invisible 


By  default,  lights  arc  white,  but  their  color  may  be  changed  by 
this  command. 

Sets  the  ambient  light  parameters. 

The  light  direction  vector  used  in  the  light  model  is  different  for 
each  point  in  the  scene. 

The  light  direction  vector  used  in  the  light  model  is  the  same  for 
all  points  in  the  scene. 

Turns  on  the  lights  and  and  makes  visible  the  figures  associated 
with  the  lights. 

Makes  the  figures  associated  with  the  lights  invisible.  Does  not 
turn  off  the  lights. 


CSG  MENMU 

union  segment 
intersect  segments 
difference  segments 

INFO  MENU 

figure  info 
segment  info 
site  info 
joint  info 
node  info 
edge  info 
face  info 
pixel  info 

show  memory  usage 
show  Jack  version 


(These  commands  don't  work.  They  are  supposed  to  perform 
constructive  solid  geometry  operations.) 


(The  following  commands  give  information  about  the  specified 
object). 
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WINDOW  MENU 

make  window  shaded 
make  window  wireframe 

make  window  ordinary 

create  shaded  window 
create  wireframe  window 
create  ordinary  window 
delete  window 
freeze  window 

thaw  window 
turn  camera  on 
turn  camera  off 
set  window  location 

make  viewer  local 
make  viewer  infinite 
set  window  ambient 
set  window  attenuation 
write  window  image 


Once  a window  is  made  shaded,  the  attributes  of  individual 
figures  and  segments  cannot  be  displayed  (i.e.  make  segment 
wireframe). 

Once  a window  is  made  wireframe,  the  attributes  of  individual 
figures  and  segments  cannot  be  displayed  (i.e.  make  segment 
shaded). 

In  an  ordinary  window,  the  attributes  of  individual  figures  and 
segments  can  be  displayed. 

Creates  a shaded  window  with  a slightly  different  view. 

Creates  a wireframe  window  with  a slightly  different  view. 

Creates  an  ordinary  window  with  a lightly  different  view. 

Deletes  a window  without  quitting  Jack. 

A frozen  window  is  not  redrawn  while  executing  the  "change 
view”  command  until  ESC  is  pressed. 

Unfreezes  a frozen  window. 

Displays  figure  associated  with  camera. 

Hides  figure  associated  with  camera. 

Prompts  user  for  window  coordinates  and  redraws  window  in 
specified  location. 

Controls  the  lighting  properties. 

Controls  the  lighting  properties. 

The  default  ambient  lighting  can  be  changed  by  this  command. 
Attenuates  lighting  values. 

Dumps  window  as  a .rle  file  for  printing. 


ORTHO  WINDOW  MENU 


four  panel  screen 
create  x window 


Creates  four  equal  sized  windows,  one  ordinary  and  three 
orthogonal. 

Creates  a x orthogonal  window. 
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create  y window 
create  z window 


Creates  a y orthogonal  window. 
Creates  a z orthogonal  window. 


BACKGROUND  MENU 
turn  background  off 

turn  background  on 

turn  grid  off 
turn  grid  on 
turn  stars  off 
tum  stars  on 

make  background  shaded 
make  background  wireframe 
set  grid  blocks 

tum  projections  on 
tum  projections  off 
tum  x projection  on 
tum  x projection  off 
tum  y projection  on 
tum  y projection  off 
tum  z projection  on 
tum  z projection  off 


Removes  the  grid  that  represents  the  ground  plane  of  the  scene 
and  the  stars  in  the  sky. 

Creates  a grid  which  represents  the  ground  plane  of  the  scene 
and  stars  in  the  sky. 

Turns  ground  plane  grid  off  but  leaves  stars. 

Creates  a grid  which  represents  the  ground  plane  of  the  scene. 
Removes  the  stars  from  the  sky. 

Places  stars  in  the  sky. 

Makes  grid  shaded. 

Displays  grid  in  wireframe. 

This  controls  the  size  of  the  ground  plane.  Prompts  user  to 
input  major  and  minor  grid  size. 

Turns  all  orthogonal  projections  on. 

Turns  all  orthogonal  projections  off. 

Turns  on  x orthogonal  projections. 

Turns  off  x orthogonal  projections. 

Turns  on  y orthogonal  projections. 

Turns  off  y orthogonal  projections. 

Turns  on  z orthogonal  projections. 

Turns  off  z orthogonal  projections. 


DISPLAY  MENU 

make  everything  shaded 
make  everything  wireframe 
make  figure  shaded 


Display  window  in  shaded  mode 
Display  window  in  wireframe  mode 
Displays  specified  figure  in  shaded  mode. 
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make  figure  wireframe 
make  segment  shaded 
make  segment  wireframe 
make  segment  transparent 
turn  segment  on 
turn  segment  off 
turn  site  on 
turn  site  off 
turn  segment  sites  on 
turn  segment  sites  off 
turn  nodes  on 
turn  nodes  off 
turn  edges  on 
turn  edges  off 
face  enumeration  on 
face  enumeration  off 
turn  segment  projection  on 
turn  segment  projection  off 
turn  figure  projections  on 
turn  figure  projections  off 

COLOR  MENU 

set  major  grid  color 
set  minor  grid  color 
set  major  highlight  color 
set  minor  highlight  color 
set  site  color 


Displays  specified  figure  in  wireframe. 

Displays  specified  segment  shaded. 

Displays  specified  segment  wireframe. 

Displays  specified  segment  as  transparent 
A tumed-off  segment  can  be  redisplayed  by  this  command. 
A turned  off-segment  is  not  displayed. 

Displays  site  as  a read  axes,  labeled  with  x,  y,  and  z. 
Turns  site  marker  of  specified  site  off. 

Display  all  sites  as  crosshairs. 

Don't  display  segment's  sites. 

Displays  nodes  as  crosshair. 

Cancels  display  of  nodes. 

Error:  This  command  turns  edges  off. 

Error:  This  command  is  not  functioning  correctly. 

The  index  of  each  face  will  be  displayed  in  the  center. 
Turns  off  display  of  face  indices. 

Turns  on  x,  y,  and  z projections  of  segment. 

Turns  off  x,  y,  and  z projections  of  segment. 

Turns  on  x,  y,  and  z projections  of  figure  (default). 

Turns  off  x,  y,  and  z projections  of  figure. 


Sets  color  of  major  grid  of  ground  plane. 

Sets  color  of  minor  grid  of  ground  plane. 

Changes  color  Jack  uses  to  indicate  picked  figures. 
Changes  color  Jack  uses  to  indicate  picked  figures. 
Sets  color  of  site  marker. 
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set  node  color 
set  wheel  color 
set  spoke  color 


Sets  color  of  node  markers. 

Sets  color  of  rotation  indicator. 
Sets  color  of  rotation  wheel  spoke. 


PARAMETER  MENU 
set  scene  scale 

set  scene  aux  scale 

set  line  width 
set  distance  units 
set  distance  precision 
set  angle  units 
set  angle  precision 
set  mass  units 
set  mass  precision 
set  rotation  type 

set  pcabody  path 
set  view  glide 
set  move  glide 
change  directory 


Default  is  100,  which  means  that  a human  figure  fills  most  of  the 
screen.  The  scale  can  be  changed  by  this  command. 

Changes  the  auxiliary  scene  scale  which  includes  sites,  nodes, 
and  rotation  wheel.  By  default,  the  setting  is  1/4  of  the  scene 
scale. 

Default  is  1.  May  be  increased. 

Legal  units  are:  mm,  cm  (default),  m,  in,  ft,yd,  mi. 

By  default,  the  precision  is  set  to  two  decimal  places. 

Legal  units  are:  deg  (default)  and  radians. 

Change  value  from  default  2 decimal. 

Legal  units  are:  g,  kg,  and  lb. 

Change  value  from  default  2 decimal. 

Prints  the  rotation  part  of  the  homogeneous  transformation  either 
in  terms  of  xyz  (default)  or  the  quat  operator. 

Sets  the  path  to  the  pcabody  library. 

Select  mouse  sensitivity  for  view  command. 

Select  mouse  sensitivity  for  move  command. 

Lets  you  change  directory  from  within  Jack. 


SIMULATION  MENU 

simulation  off 
set  simulation  level  1 

set  simulation  level  2 
single  set  simulation 


Simulation  turned  off  completely. 

Default.  Jack  evaluates  the  simulation  function  only  during 
object  manipulation:  moving  figures,  adjusting  joints,  etc. 

Continuous  evaluation  of  the  simulation  function. 

Simulation  stops  when  minimum  of  function  is  found. 
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read  squash 
generate  scaling  squash 
bind  joint  to  port 


Define  deformation  matrix. 

Lets  you  deform  figures. 

Waits  for  a specific  connection  on  a specific  port. 


RETINA  DISPLAY  MENU 


create  retina  window 

create  field  of  view  cones 

fixate  eyes  on  site 

interactive  fixation 
monocular/binocular  map 

zoom  in/out  of  retina  window 
show  eye  scan 


Creates  a window  displaying  a retina,  with  the  view  projected 
onto  it  in  retinal  coordinates. 

Prompts  for  figure  selection.  Draws  60°  field  of  view  cones 
from  each  eye  of  the  selected  figure. 

Fixates  the  view  to  a specified  site.  Draws  lines  from  each  eye 
to  the  site. 

The  fixation  point  can  be  moved  interactively. 

By  default,  the  retina  window  shows  the  images  perceived  by 
both  eyes.  To  simplify,  a monocular  view  can  be  displayed. 

Zoom  toggle. 

Trace  of  eye  scan. 


RETINA  EDITOR  MENU 


create  retina  editor 

draw  left  retina  object 

draw  right  retina  object 

draw  filled  retina  object 

retroject  left  retina 

retroject  right  retina 

retroject  both  retina 

clear  retina  objects 

save  retina  objects 

load  retina  objects 

zoom  In/Out  of  editor  window 


Creates  a retina  window  into  which  user  may  draw. 
Accepts  user  input  for  right  retina. 

Accepts  user  input  for  left  retina. 

Displays  retina  object  in  shaded  mode. 

Object  of  left  retina  is  projected  into  3-dimensinal  space. 
Object  of  right  retina  is  projected  into  3-dimensional  space. 
Objects  of  both  retina  is  projected  into  3-dimensional  space. 
Clears  retina  window 
Writes  coordinates  of  retina  objects  to  file. 

Reads  objects  into  retina  editor  window  from  file. 

Zoom  toggle. 
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VISION  INFORMATION  MENU 


create  adaptation  info  window 

create  legend  window 

create  Aitoff  window 
initialize  state  info 
set  state  information 


Creates  a window  that  gives  information  about:  retinal  map 
orientation,  field  of  view  objects,  and  fixation  distance. 

Creates  a legend  window  that  gives  information  about  color 
codes  etc. 

Creates  an  Aitoff  window  with  the  view  from  a specified  figure. 

Initializes  the  vision  parameters  to  default  values. 

Creates  a meter  window  that  allows  to  adjust  clipping  planes, 
lumination  and  illumination  parameters. 


TRACE  MENU 
trace  site 

untrace  site 
trace  segment 
untrace  segment 
set  trace  color 
clear  trace 
delete  trace 


Creates  a trace  of  the  site  if  the  segment,  containing  the  site,  is 
moved. 

Turns  off  tracing  of  site  but  leaves  trace  intact 
Creates  a trace  of  the  segment  if  the  segment  is  moved. 

Turns  off  tracing  of  the  segment,  but  leaves  trace  intact. 

The  color  of  an  existing  trace  can  be  changed. 

Clears  trace  without  disabling  it 
Deletes  trace  and  turns  off  tracing. 


COMMAND  MENU 

help  for  command 
help  by  subject 
key  mode  on 
keymode  off 
disable  WM  quit 
disable  graphics 
enable  graphics 
bind  command  to  key 


We  don't  have  on-line  help. 

We  don't  have  on-line  help. 

When  turned  on,  expects  keyboard  input 

Returns  mode  to  default 

Prevents  user  from  accidentally  quitting  jack. 

Window  will  not  be  drawn  until  executing  "enable  graphics". 

Redraws  window  after  it  has  been  disabled. 

After  a command  is  bound  to  a key,  the  command  can  be 
executed  by  pressing  the  specified  key. 
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describe  key  bindings 
create  communication  port 
send  to  communication  port 
close  communication  port 

ANIMATION  MENU 
help 

describe  script 
initialize  animation 

create  key  frame  from 
current  state 

delete  keyframe 

change  key  frame  to 
current  state. 

change  keyframe  normal  time 
change  frame  rate 

GROUP  MENU 

create  figure  group 
create  figure  joints  group 
create  figure  location  group 
create  camera  group 
create  joint  group 
create  segment  nodes  group 
create  mixed  group 
change  group  name 
describe  group 


Displays  a list  of  all  the  key  bindings. 

Waits  for  a connection  by  a client  program  to  be  established. 
Lets  user  send  message  to  communication  port 
Closes  a communication  port 


Prints  instructions  for  using  the  animation  menu. 

Prints  script  on  screen 

This  command  needs  to  be  called  once  before  the  animation 
process  can  be  started.  For  simple  animation,  the  default  values 
may  be  accepted. 

A key  frame  is  generated  as  part  of  the  animation  script.  As  a 
keyframe  normal  time,  choose  0 for  first  and  1 for  last  keyframe 
and  values  such  as  0.2 , 0.6  in  between. 

Removes  a keyframe  from  the  script. 

Changes  a keyframe  from  the  script 


Lets  you  changes  the  normal  time  of  a keyframe 
The  default  is  30  frames  per  second. 


Uses  all  joints  of  a figure  plus  its  location  (root). 
Uses  all  joints  of  a figure. 

Uses  a figure's  location  (root). 

Uses  the  camera's  location. 

Uses  a single  joint. 

Uses  nodes  of  a segment  (Used  for  deformation) 
May  use  any  number  of  members  of  the  above  type. 
Replaces  old  group  name. 

Prints  all  group  members  on  screen. 
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delete  group 


A group  may  be  deleted. 


ACTION  MENU 

choose  current  action 
create  action 
change  action  length 

change  action  start  time 
change  action  interpolation 
change  action  name 
delete  action 


PLAYBACK  MENU 

step  through  key  frames 

Lets  you  look  at  the  key  frames  you  have  created. 

playback 

Single  playback  of  animation. 

step  through  interpolated 
frames 

Steps  through  the  in-between  frames  the  program  has 
generated. 

repeat  playback 

Continuous  playback  of  the  animation. 

playback  in  real  time 

Playback  speed  may  not  be  consistent. 

special  playback 

Use  for  squashed  objects. 

read/write  RECORD  MENU 

read  script  file 

Reads  a file  with  a .scr  suffix. 

read  action  file 

Reads  a file  with  a ACT 

write  script  file 

Generates  environment*  script,  and  action  files. 

write  action  file 

Generates  action  file  only. 

save  complete  environment 
files  for  rendering 

Generates  environment  files  of  key  frames. 

save  incremental  environment 

Generates  environment  files  of  incremental  frames. 

Current  action  is  the  most  recently  created  action. 

Create  a new  action. 

The  action  time,  which  was  set  by  "initialize  animation,  may  be 
changed  by  this  command. 

With  multiple  actions,  the  start  times  can  be  changed. 

Choose  between  spline  (default)  and  linear. 

This  does  not  create  a file. 

An  action  can  be  deleted. 
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files  for  rendering 

record  animation  Recording  animation  on  video. 
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APPENDIX  C 


JACK  DIRECTORY  STRUCTURE 
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APPENDIX  D 


ANTHROPOMETRY  DEMO  1990 
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ANTHROPOMETRY  DEMO  1990 


INTRODUCTION  OF  ANTHROPOMETRY  ANALYSIS  BY  B.  SMITH: 

One  of  the  most  fundamental  requirements  for  MIDAS  is  a physical 
representation  of  the  human  operator  to  explore  their  interaction  within  the 
geometry  of  a crew  station  or  aircraft.  To  meet  this  need  we  have  Jack,  a 
dynamic  anthropometric  model  developed  through  a grant  with  Dr  Norm 
Badler  at  the  University  of  Pennsylvania.  Jack  is  an  interactive  graphic 
package  that  allows  the  creation  and  manipulation  of  3-D  human  figures, 
both  statistical  and  individual,  in  a 3-D  object  space,  allowing  a designer  to 
explore  reach,  fit,  and  human  motion.  Christian  Neukom  will  demonstrate 
several  of  the  features  of  this  model  and  following  him  will  be  a discussion 
by  Dr.  Norm  Badler  from  UPenn,  to  describe  a little  bit  about  where  he 
intends  to  take  Jack  in  the  Future. 

INITIAL  SETUP: 

open  window;  godl;  stow 
open  window;  god2;  stow 
open  window;  god3;  stow 
open  window;  god5;  stow 

START  DEMO: 
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DEM01  (Display  of  a selection  of  polybody  and  biostereometric  figures) 

P°P  f Jack  iscapable  of  displaying  and  manipulating  a range  of  eitherindividuaUy 
defined  or  statistical  body  figures.  Here  I m showing  you  a selection  of 
polybody  figures  including  the  5*.  50*  and  95*  percentile  male  and 
female  based  on  the  NASA  Aircrew  Demographics  for  the  Year  2000 
contained  in  NASA  std  3000.  Also  shown  is  a 95*  percentile  by  height 
biostereometric  male  figure.  Associated  with  these  figures  is  a great  deal  of 
anthropometry  data  regarding  segment  size,  strength,  center  of  mass,  and 
joint  limits,  which  I will  go  into  more  detail  on  later.  What  I want  to  show 
you  now  is  a couple  of  the  new  features  of  Jack. 

quit  jack  window 

DEM02  (Female  polybody  figure  is  set-up  to  do  balanced  reach 

male  polybody  figure  has  two  hands  constrained  to  a box) 

pop  window2 

do  balanced  reach  (ready  to  go,  just  move  mouse) 

One  of  the  new  features  of  Jack  is  balanced  reach.  I m demonstrating  it 

with  an  interactive  reach  toward  the  floor.  As  you  can  see,  the  mannequin 
uses  the  opposite  leg  to  balance  its  body. 

hit  Esc  to  quit  reach  . 

move  figure  (box)  (up/down  and  forward/backward  only) 

Another  addition  is  the  facility  of  the  various  reach  constraints.  The 
mannequin’s  hands  are  constrained  to  the  sides  of  the  box.  As  I move  the 
box  the  hands  follow  simulating  handling  of  the  box.  This  feature  is  useful 
when  the  mannequin  interacts  with  a maintenance  ?n^t°^e,nt 
I have  turned  on  the  site  tracing  on  the  mannequin  s left  hand.  The  trace, 
shown  as  a yellow  line,  shows  us  the  exact  path  and  indicates  if  the  hand 
went  through  any  objects  since  there  is  no  built-in  collision  detection  and 

avoidance. 

quit  jack 


Page  G-51 


DEM03  (Co-pilot  Gunner  crewstation  with  polybody  figure  doing  a reach  with 
the  right  hand  to  the  left  Multi  Functional  Display  in  an  animation  script. 
A ’figure  Info*  command  has  been  inserted  into  the  script  to  prevent  the 
animation  from  executing  - an  Esc  will  start  the  animation) 

Jack  can  be  used  to  explore  human  figure  interactions  in  a 3-D  space 
environment  either  interactively  as  I am  doing  now.  through  animation 
script  files,  or  controlled  through  our  integrated  simulation  as  you  will  see 
later.  What  I'll  do  now  is  import  one  of  our  Longbow  crewstation  files 
created  by  S Systems  from  Apache  source  data  and  execute  a few 
movements. 

pop  window3  (reach  animation) 

I've  loaded  the  Co-pilot  Gunner  crewstation,  with  several  extraneous 
portions  deleted  to  speed  up  graphics  processing.  With  the  new  Longbow 
cockpit  changes,  one  of  the  questions  a designer  may  want  to  answer  is  if 
the  two  Multi  Functional  Displays  can  comfortably  be  used  with  either 
hand,  since  a great  many  cockpit  functions  arc  enabled  by  activating  soft- 
switches  on  the  MFD  touch  screens.  I've  placed  a 95th  percentile  figure 
into  die  seat  Mid  as  you  can  see,  a rather  large  Optical  Relay  Tube  protrudes 
significantly  from  the  front  panel  in  between  the  left  and  right  MFDs 
hit  Esc  to  start  animation 

^ J?ow  mn  80  *niination  script  that  I previously  set  up  to  see  if  the  Co- 
pilot  Gunner  can  reach  the  left  MFD  with  his  right  hand  I have  constrained 
his  left  hand  to  the  collective  and  Tm  allowing  him  to  move  through  his 
wctJ*"50  sca^^  inertial  reel  is  not  locked*  Now  generally,  the  left 
MFD  is  operated  with  the  left  hand  and  the  right  MFD  with  the  right  hand 
but  there  may  be  situations  when  this  is  not  possible  - for  example  if  one  of 
the  hands  is  busy  with  another  task  or  one  of  the  MFDs  is  out  of  order  We 
can  conclude  from  watching  this  animation  that  it  is  fairly  cramped  quarters 
and  the  Co-Pilot  Gunner  has  to  contort  a bit  to  perform  this  task 
quit  jack 
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DEM04  (Biostereometric  figure  wearing  IHADDS  helmet,  visor  and  monacle. 

Figure  is  displayed  in  3-D  window  and  in  3 orthogonal  windows) 

90d 4 Another  one  of  the  new  fentures  of  Jack  concerns 

the  Dolvbody  figures  are  quick  to  generate  and  manipulate.  Dr.  Badler  has 
incorporated  some  biosteriometric  scanned  body  images  Amorce 

whiclrprovide  a more  pleasing  graphical  image  together  with  H^detmlsm 
certain  segments  of  the  body  such  as  thetoreo  and  head.  These  o e 

of  which  is  shown  here,  contain  over  6000  data  points  each  and  arebased 
on  actual  subjects  from  three  different  somatotjyes  in  each  se^  I am  also 
showing  an  Apache  IHADDS  helmet,  visor,  and  monacle  with  this  figure 
since  we've  received  a number  of  questions  about  wether  the^fects  of 
protective  clothing  or  flight  gear  can  be  represented  in  Jack.  The  ttaee 
orthogonal  windows  are  also  a new  feature,  and  are  extremely  han  y 
up  objects  such  as  this  helmet  for  proper  placement. 

move  allows  y0U  t0  turn  individual  portions  of  figures  into  wireframes 

or  transparent  as  shown  in  the  visor. 

quit  Jack 

DEM05  (Apache  Longbow  helicopter  with  two  biostereometric  figures  seated  in 
crew  position) 

Before'l  lwtveJack,  111  bring  up  a window  showing  you  the  complete 
Apache  Helicopter  with  two  Biostereometric  figures  seated  in  thecrew 
positions.  While  impressive  to  look  at.  this  figure  has  about  10  000 
polygons,  and  it  unfortunately  slows  down  the  graphic  processing 
enormously,  especially  in  shaded  mode.  As  these  hardware  platforms 
evolve,  perhaps  we  will  be  able  to  easily  manipulate  environments  as 
complex  as  this  one. 
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DEM06  (Spreadsheet  Anthropometry  Scaling  System,  SASS) 

go  through  poster  first 

god6 


Displayed  here  on  the  screen  is  the  anthropometry  spreadsheet  displaying  the 
regnrent  dimensions  of  the  50*  percentile.  Other  anthropometric  groups  can  easily 
be  selected  to  view  or  modify  the  data.  Below  is  a summary  of  the  data  being 
delayed  to  allow  the  user  to  have  a "global"  view  of  the  figure  he  is  working  on. 
Presently  there  are  eleven  labels  defined:  population,  gender,  mass,  stature,  group 

selec, ““■*  “•  ***» 
select  Group  %ile  and  change  to  new  %ile 

Let  s look  at  an  example.  Let's  say  we  want  to  find  out  what  percentile  figure  can 
wear  a certain  sized  helmet  We  simply  input  the  dimension  (125")  in  the8”™ 
respective  cell  and  read  the  corresponding  percentile.  We  can  then  change  the 
group  percentile  to  display  the  other  dimension  of  that  group. 
click  on  Query  -->  Database  Query  Screen 
click  on  Input  Data;  from  keybord  enter:  people.db 
click  on  Query  Database  and  build  query: 
male,  stature  > 160  and  stature  < 170,  done 

L«t  us  now  turn  our  attention  to  the  database  query  spreadsheet  First  I'm  going  to 

nlh i?HaEU  dlt!base-  V 1 s *?£  wc  w?nt  10  know-  if  there  are  any  indmduals 
in  this  database  who  arc  male  and  have  a height  of  160  - 170  cm.  The  query  is 

conveniently  built  through  menu  selection.  We  come  up  with  3 individuals  who 
satisfy  these  requirements.  The  data  of  these  individuals  could  now  be  displayed  in 
the  anthropometry  spreadsheet  and  their  figures  displayed  in  Jack  7 


Summary 

ii  , . This  concludes  the  anthropometry  presentation.  I had  only  time  to  show  vou 

a small  selection  of  the  features  of  Jack  and  SAS$.  If  you  like  to  know  wy  othw  feaS 
or  more  details  about  the  ones  I presented,  please  come  to  me  in  the  individual  discussion 

session  after  the  formal  demo. 
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APPENDIX  E — README  FILE 
Instructions  for  Installing  Jack  from  tape. 
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INSTALLING  THE  JACK  SOFTWARE 


Create  a directory  where  you  want  to  copy  your  files  to,  i.e. 

mkdir  /usr/upenn  (if  you  are  super  user) 
or  mkdir  upenn  (if  you  are  in  your  home  dir) 

Remember  that  you  need  41  Mb  of  disk  space. 


Read  the  tape  into  this  directory,  i.e. 
tarxvo 

When  the  software  is  extracted  from  the  tape,  it  will  create 
the  following  directories: 


4.5 

contains  source  files 

4D 

contains  executable  files 

gen 

contains  figure  and  other  data  files 

sass 

contains  files  used  by  the  Spreadsheet 
Anthropometric  scaling  system 

Jackshell 

Source  this  shell  to  set  up  environmetal 
variables  used  by  Jack  and  sass. 

Readme 

This  file 

EditdneJ^kshell  and  change  the  UPENN  environmental  variable. 
UPENN  contains  the  absolute  path  from  the  root  to  the  directory 
into  which  you  copy  the  files  from  the  tape.  Find  the  line 
setenv  UPENN  /usr/upenn"  and  change  the  path. 

After  making  the  changes  source  the  Jackshell  to  set  all  the  paths: 

source  Jackshell 

To  run  Jack,  change  to  the  4.5/longbow  directory  and  run  Jack: 
cd  4.5/longbow 

jack  vis.jcl  (vis.jcl  is  an  environment  example) 


If  Jack  runs  but  cannot  find  the  files  it  needs  to  load,  check 
the  path  set  in  UPENN.  If  it  runs  and  the  crashes,  it  might  be 
because  of  a different  version  of  the  operating  system.  In  this 
case,  try  to  relink  Jack  by  executing: 
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cd  $UPENN/4.5/gen/src/jack 
make 


if  there  are  still  problems,  try  to  recompile  the  complete  program  by 
executing  the  following  commands: 

cd  $UPENN/4.5/gen/src/lib 
make 

This  command  will  execute  a makefile  in  each  of  its  subdirectories. 
After  the  make  completes  run  the  command 

SUPENN/4. 5/gen/src/jack/make. 


If,  after  undertaking  all  of  these  steps,  you  have  still  problems, 
please  contact  Mike  Prevost  at  E-mail  address: 

prevost@eos.arc.nasa.gov 


or  Christian  Neukom  at: 

neukom@eos.arc.nasa.gov 


To  run  sass,  change  to  the  sass  directory  and  type  sass. 
If  it  doesn't  run,  relink  it  by  executing  make  m the  sass 
directory. 
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Annex  H 


Army-NASA  Aircrew/Aircraft  Integration  Program:  Phase  IV 


A H 


Man-Machine  Integration  Design  and  Analysis  System  (MIDAS) 
Software  Detailed  Design  Document 

Vision  Models 


prepared  by 
Michael  Prevost 
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1.0  INTRODUCTION 

1.1  identification  of  document 

This  is  the  Software  Product  Specification  for  the  Vision  analysis  modules  of  the  Mai- 
machine  Integration  Design  and  Analysis  System  (MIDAS).  Included  is  docunwnmuonfor 
the  David  Samoff  Legibility  Model  (LM),  the  Lighthouse's  Volume  Penmetry  M^el  <y 
and  die  Vision  Enhanced  Jack  Interface  ( VEJI).  These  three  components  combine  to  form 
the  MIDAS  vision  model.  Documentation  for  the  standard  Jack  software  is  documented  in 
KaS!  model  ( inside  the  circle  area ) is  shown  in  the  context  of  the  enttre 

MIDAS  system  in  Figure  1. 
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1.2  SCOPE  OF  DOCUMENT 


The  high  rate  of  change  of  features  and  the  exploratory  nature  of  the  vision  model  makes 
documentation  a difficult  task.  It  should  be  emphasized  that  the  software  that  this  document 
pertains  to  is  still  under  development.  What  this  document  gives  the  reader  is  a snapshot  of 
the  current  state  of  a continuously  evolving  software  package. 

Because  the  software  is  under  development  there  has  been  little  effort  made  to  bullet  proof 
the  interface  and  no  attempt  to  optimize  the  code.  There  are  functions  that  are  incomplete 
and  may  not  work  well  due  to  their  early  phase  of  development  There  are  ways  to  view 
and  plot  data  that  may  lead  the  user  to  erroneous  conclusions.  Most  of  the  Hata  presented  by 
the  vision  model  has  not  been  validated.  There  are  known  inaccuracies  in  the  mapping 
functions.There  will  be  no  attempt  made  to  stay  upwardly  compatible,  consistent  with,  or 
many  of  the  other  conventions  that  the  user  may  expect  from  a more  mature  commercial 
software  product.  The  ability  for  me  to  support,  answer  questions  or  help  users  may  be 
very  limited.The  next  phase  of  development  will  involve  substantial  changes  to  the 
software  architecture  of  the  model. 

In  short  this  is  developmental  software  and  not  production  software.  The  following 
standard  disclaimer  applies:  there  are  no  warranties,  either  expressed  or  implied,  regarding 
the  enclosed  computer  software  package,  its  merchantability,  or  its  fitness  for  any  particular 
purpose. 


1.3  PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 

This  document  is  intended  to  convey  to  the  user  the  class  of  vision  phenomena  supported 
by  the  model,  and  how  he/she  can  vary  the  parameters  of  the  vision  model  in  order  to 
answer  crewstation  design  related  questions.  The  software  requirements  and  rationale  are 
explained  and  the  limitation  and  tradeoffs  of  the  models  enumerated. 

2.0  RELATED  DOCUMENTATION 


2.1  APPLICABLE  DOCUMENTS 

The  following  documents  are  referenced  herein  and  are  directly  applicable  to  this  volume: 

A.  Arditi,  The  Volume  Visual  Field:  A Basis  for  functional  perimetry.  Clinical  Vision 
Sciences,  3 ( 1988). 


A.  Arditi,.  Alternate  Representation  of  Visual  Space,  SPIE  Vol.  1083  Three  dimensional 
Visualization  and  Display  Technologies.,  242-245,1989. 

C.A.  Curcio,  et  al.,  Human  Photoreceptor  Topography,  The  Journal  of  Comparative 
Neurology  292:497-523  1990. 

C.R. Carlson  and  R.W.  Cohen,  A simple  Psychophysical  Model  for  Predicting  the 
Visibility  of  Displayed  Information,  Proceedings  of  the  SID , 21, 229-246  ( 1980). 

J.  Larimer,  et  al,  A Computer-Aided  Tool  for  Assessing  the  Visibility  of  Cockpit 
Displays,  Society  For  Information  Display  90  Digest  133-135,  1990. 

E Adelson,  C.  Carlson,  A.  Pica,  Modelling  the  Human  Visual  System,  RCA 
Engineer  27-6  Nov./Dec.  1982. 
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E Adelson,  et  al,  Pyramidal  Methods  in  Image  Processing,  RCA  Engineer  29-6 
Nov./Dec.  1984. 


C Hall,  E.  Hall,  A Nonlinear  Model  for  the  Spatial  Characteristics  of  ^u™pn7  N 
Visual  System,  IEEE  Transactions  on  Systems,  Man,  and  Cybernectics,  Vol.  SMC-  , 

3,  March  1977. 


R.  Hall,  Illumination  and  Color  in  Computer  Generated  Imagery,  Springer- Verlag,  New 
York,  New  York,  1989. 


IES  Lighting  Handbook  , 1984  Reference  Volume 


2.2  INFORMATION  DOCUMENTS 

The  following  documents  amplify  or  clarify  the  information  presented  in  this  volume: 

R.Blake  and  R.  Fox,  The  psychophysical  inquiry  into  binocular  summation, 

Perception  and  Psychophisics,  14,  161-185  (1973). 

Bouma,  H.  Visual  recognition  of  isolated  lower-case  letters.  Vision  Research  II,  459- 
474  ( 1971). 

Coffin,  S.  Spatial  frequency  analysis  of  block  letters  ^s  not  Pred,ct  exPenmental 
confusions.  Perception  and  Psychophysics  23(1),  69-74  ( iy»l). 

Egeth,  H.E.,  and  Santee. J.L.  Conceptual  and  perceptual  components  of  interletter 
inhibition.  Journal  of  Experimental  Psychology  7(3),  506-517  1981. 

B.  Esterman,  Functional  scoring  of  the  binocular  field,  Opthalmology,  89,  1226-1234. 

Gervia.M.J.,  Lewis  O.  Harvey,  and  Roberts,  J.O.  Idntificationconfusions 
among  letters  of  the  alphabet.  Journal  of  Experimental  Psychology  10(5)  655-666  198 

Geyer,L.H.,  and  DeWald,  C.G.  Feature  list  and  confusion  matrices  . Perception  and 
Psychophysics  14(3),  471-482  1973. 

Geyer,  L.H.  and  Gupta,  S.M.  Recognition/confusion  of  dot  matrix  vs.  conventional 
font  capital  letters.  Perception  and  Psychophysics  29 , 280-282,  1981 

Gilmore, G.C.,Hersh,H.,Caramazza, A.,  and  Griffin,  J.  Multidimensional  letter 
similarity  derived  from  recognition  errors.  Perception  and  Psychopysics  25, 425-431, 

1979. 

Heiiden,  A.H.C.,  Malhas,  M.S.,  and  Roovaart,  B.P..  An  empirical  mterletter 
confusion  matrix  for  continuous-line  capitals.  Perception  and  Psychophysics  35(1),  85-88, 
1984. 

Mewhort,  D.J.K.,  and  Dow,M.L..  Multidimensional  letter  simUarity:  A confound 
with  brightness?  Perception  and  Phychophysics  26(4),  325 -51b,  iy/v. 

Morrison,  I. A LISP  program  to  determine  similarity  relations  in  letter  displays.  Behavior 
Reseach  Methods  and  Instrumentation  15(1),  69-71,  iyo3. 
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Townsend,  J.T.,  Theretical  analysis  of  an  alphabetic  confusion 
Phychophysics  9(  1 A),  40-50,  1971. 


matrix.  Perception  and 


Watson,  A.B.,  pe  ideal  observer  as  a modelling  tool.  Frontiers  of  visual  science: 
Proceedings  of  the  1985  Symposium  ( pp.  32-37).  Washington  D.C.:  National  Academy 

15(l),°A3f,‘?986nd  Fitzhugh’  A,E”  A new  look  at  ,cttcr  recognition.  Perception 

Watson,  A.B.,  and  Fitzhouh,A.E.,  Modelling  character  legibility.  Society  for 
Information  Display  Digest  of  technical  Papers  20,  360-363, 1989. 

3.0  CONCEPT 

3.1  DEFINITION  OF  VISION  MODELS 

3.1.1  PURPOSE  AND  SCOPE 

In  a crewstation  (e.g.  an  aircraft  cockpit)  the  ability  of  the  operator  (pilot)  to 
unambiguously  perceive  rapidly  changing  information  both  internal  and  external  to  the 
Sstfon  ,s  <?ntlcak  To  ^ess  the  impact  of  crewstation  design  decisions  on  the  pilot’s 
ability  to  perceive  information,  the  designer  needs  a means  of  evaluating  the  trade-offs  that 

MomefHSl^h-eretntKd?fgnS‘  There  exist  -many  CAD  tools  for  creating  and  manipulating 
geometncal  objects  but  few  contain  any  vision  capabilities.  Of  the  ones  that  do  have  the  8 

nlnt«yfi?iHd<f* -S  SOme  V1SUal  of  design,  only  the  geometrical  questions  eg.  vision 

K’,rield,°f:iew’  Vc' OF  answcrcd-  0thcrs  ( Watson,  BergenJHall)  have  developed 
tools  to  predict  visual  performance  but  they  have  never  been  integrated  with  a CAD-like 
tool.  Because  of  the  interactive  nature  of  design  it  is  necessary  to  provide  feedback  on 
visua  perfoimance  as  a continuum  of  design  parameters  are  varied.  Without  integration  the 
usefulness  of  vision  models  to  the  design  community  is  gready  limited. 

TTie  process  of  crewstation  design  involves  a large  number  of  tradeoffs.  The  purpose  of  the 
vision  model  is  to  provide  an  easy  to  use  interactive  interface  to  vision  relevant  parameters 
of  crewstation  design,  and  through  data  visualization  techniques  provide  feedback  to  the 
designer  on  the  effects  of  varying  these  parameters  to  the  appropriateness  of  the  design. 

3.1.2  GOALS  AND  OBJECTIVES 

3kfr.nnr^i^H8^al  °f  visk>n  .‘s  s™ply  to  explore  ways  to  better  communicate 
* S?difSS1?n  1SfesTd  Pr?vulde  ^formation  for  design  changes  that  would  result  in 
a better  overall  design.  It  is  beyond  the  capabilities  of  the  current  vision  model  and  the 
scope  of  our  simulation,  to  predict  when  all  of  the  necessary  conditions  are  present  for 
proper  perception  to  occur.  The  goal  of  the  vision  model  is  to  show  only  when  it  is  at  least 
possib  e to  perceive  the  stimuli  and  in  the  case  of  MFD  characters  their  probability  of 
correct  discrimination.  Some  primary  areas  of  focus  will  be  on  character  legibility  as 

HMa  VnK  °n,  *,£  MFD’ field  of  view  volumes  and  intersections,  performance 

data  linking  to  the  CAD  representation  of  the  design  and  a polar  foveal  centric 

representation  of  objects  in  the  design. 
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3.2  USER  DEFINITION 

The  vision  model  assumes  that  the  user  is  an  experienced  c^wstation  desi^er  but  not 
necessarily  adept  at  applying  human  factors  principles  to  all  aspect  of  the  design.  It  is 
SScted  that  the  designer  is  using  this  tool  in  an  iterative  fashion  to  alter  the  design  to  meet 
requirements  and  still  be  consistent  with  human  factors  guidelines. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

The  vision  enhancements  to  the  Jack  software  provide  the  designer,  for  the  first  tune,  an 
integrated  environment  featuring  CAD  like  capabilities,  an  anthropomemc ; model  md 
visual  performance  model.  Inside  this  environment  an  entire  spectrum  of  design  trade  offs 
can  easily  be  explored.  Through  a simple  interactive  interface,  a designer  can  manipulate 
design  parameters  such  as  cockpit  layout  geometry,  environmental  factors  such  as  ambient 
lighting,  pilot  parameters  such  as  point  of  regard,  and  equipment  parameters  such  as 
display  size  and  contrast  of  displayed  symbology.  These  visibility  options  provide  an  end 
to  end  i analysis  that  answers  questions  such  as  "Is  it  possible  for  the  pilot  to  read  the 
display?". 

Visual  performance  data  can  be  projected,  in  the  form  of  3D  contours,  into  the  crewstauon 
graphic  model  providing  the  designer  with  a footprint  of  the  operator  s visual 
characteristics  given  the  current  parameter  settings. 

In  Appendix  A the  figured  labeled  Binocular  Vision  Model  provides  a good  overview  of  the 
of  the  complete  vision  model. 


3.4  SAMPLE  OPERATIONAL  SCENARIOS 

The  following  is  a sample  scenario  intended  to  demonstrate  how  one  might  typically  use  the 
vision  model.  The  scenario  has  three  parts:  First,  an  example  of  how  to  use  the  designer  s 
interface,  second,  the  retinal  window  projections,  and  thirdly,  the  retinal  editor,  including 
the  use  of  discrimination  data. 

You  must  first  follow  the  standard  startup  procedure  for  Jack.  The 

will  vary  according  to  the  particulars  at  your  installation  site  but  would  look  something  like 
the  following: 


1)  Initial  setup: 

cd  /usr2/neukom2/4.5 
source  sshell  ( cshell  if  on  coral) 
cd  longbow2 
djack  vis.jcl 


This  should  start  up  the  proper  version  of  Jack  and  bring  up  a subset  of  the  pilot  crew 
Marion Swffl  put  Jack  where  and  how  he  belongs.  Die  first  example  shows  how  to  vary 
the  fixation  point.  This  must  be  done  in  order  for  other  options  to  make  sense.  An  example 
of  view  cones  that  are  drawn  down  the  newly  changed  visual  axes  is  given  for  further 

clarity. 


2)  Demonstrate  the  User  interface: 

select  ->  options  menu:retinal  display: 

select  ->  jack 

select  ->  site  on  left  MFD 


fixate  eyes  on  site 
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click  left  mouse  button  again  to  show  the  visual  axis 
press  esc  key 


select  ->  options  menu:retinal  display:  field  of  view  cones 

select ->  jack 

^°V  ^0r  Cone  ( mcssaSe  window  or  mouse) 

WARNING:  These  cones  can  not  be  deleted  after  you  create  them  without 
crashing  the  system. 


select  options  menu. retinal  display:  interactive  fixation 

select ->  jack 

move  fixation  point  via  mouse 

WARNING:  eyes  are  joint  limited.  You  will  see  the  lines  representing  the  visual 
axes  stop  moving  when  you  reach  a joint  limit.  Also  an  out-of-limit  message  above 
the  fixation  distance  and  angle  information  (lower  left  comer)  will  be  displayed. 


Next  an  example  of  how  to  use  the  retinal  window  is  given. 
3)  Demonstration  of  retinal  window: 


select->  options  menu:retinal  display:  create  retinal  window 

select->jack 

open  window  in  upper  left  comer. 

WARNING:  This  window  only  redraws  when  information  is  changed  in  the 
window.  If  you  move  the  window  it  will  not  redraw  until  you  make  a menu 
selection  that  changes  what  is  displayed  in  it.  I usually  overlay  it  on  iod  of  the 
retinal  window  to  conserve  screen  space.  v 

select-  >option  s menu:retinal  display:  Monocular/binocular 

select->option:vision  information:  set  pilot  state  open  window 

set  near  dipping  to  a small  value  -10  by  clicking  right  mouse  button  near  bottom  of 
the  near  slide  bar. 

setfar  clipping  to  -300  by  clicking  right  mouse  button  near  top  of  the  "far"  slide 
press  esc  key. 


select->options  menu:retinal  display: 
select  ->  Jack 

move  fixation  point  around  via  mouse 
press  esc  key. 


interactive  fixation 


The  retinal  editor  is  the  primary  means  by  which  performance  data  is  introduced  in  Jack 
Known  data  can  be  drawn  directly  on  the  editor  and  then  saved  to  a file.  The  file  can  then 
!n  at  aKiater  Dme  when  that  data  is  desired.  For  ease  of  demonstration,  some  menu 
selections  have  been  created  so  that  the  user  can  load  a file  directly,  without  having  to  type 

iJ/ch^i!6  °f  thC,  I® ' In,fJlera1’  is  not  the  case-  When  the  user  desires  to  load  a file  that 
he/she  has  created  it  will  be  necessary  to  type  the  name  of  the  file  before  loading  it  into  the 

oj  aicm* 


4)  Demonstration  of  retinal  editor: 
Drawing  of  an  arbitrary  object: 
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create  retinal  editor 


select->options  menus:retinal  editor: 
open  window  in  upper  right  part  of  screen 


select->options:retinal  editor:  draw  right  retina 

draw  in  retinal  editor  by  moving  the  mouse  and 
pressing  the  left  mouse  button,  end  drawing  by 

in  the  nain.  ednor  as  yon  dn™  in  i,  on  any  of  ihc 

GTX  machines, 
select- >options:retinal  editor: 


clear  retinal  objects  Loading 
and  retrojecting  published  data: 


select->options  menus:retinal  editor: 
select->options  menus:retinal  editor: 
select->options  menus:retinal  display: 
select->Jack 

move  fixation  point  via  mouse 
select->options  menus.retinal  editor: 

Sarnoff  Confusion  data: 

select->options:retinal  editor: 
select->options  menus:retina]  editor: 
select->options  menus:retinal  editor: 


load  cone  density  data 
retroject  both 
interactive  fixation 


clear  retinal  objects 


load  OQ  confusion  data 
zoom  editor  window 
draw  filled  retinal  objects 


select->options  menus:vision  information:  set  pilot  state  open  window 

set  ill  to  high  value  ( > 4)  by  clicking  near  the  top  of  the  slide  bar. 

KC^pktions  menus:retinal  editor:  clear  retinal  objects 

sdSl^Ss  menus:retinal  editor:  load  OQ  confus.on  data 

4.0  REQUIREMENTS 

4.1  REQUIREMENTS  APPROACH  AND  TRADEOFFS 

Geometrical  data  such  as  the  volume  field  of  view,  occlusions,  facial  geometry  and  helmet 
model  acceptable. 

fixauon  «L  and 

visibility  imposed  by  retinotopic  processing. 
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The  performance  contours  shall  be  generated  based  on  a discrimination  model  that  takes  as 
input  the  aviators  adaptive  state,  environmental  factors  and  stimulus  description  and 
generates  iso-discrimination  contours.  These  performance  data  must  be  mapped  back  into 
the  environment  and  linked  to  their  stimuli. 


4.2  HARDWARE  ENVIRONMENT 


The  three  software  components.  Volume  Perimetry  (VP),  Legibility  Model  (LM)  and 
Vision  Enhanced  Jack  Interface  (VEJI)  can  be  run  as  an  integrated  program  on  the  Silicon 
Graphic  4D  senes  of  workstations.  These  workstations  have  double  buffered,  24  bit  color 
o bits  alpha  and  Z buffered  frame  buffers.  The  workstations  have  hardware  support  for 
light  models  and  viewing  transformations.  The  4D  220  GTX  ( MIDAS'  fastest 
workstation)  has  two  RS2000  25MHZ  CPUs  for  a total  of  40  MIPS.  The  software  makes 
use  of  only  one  CPU  at  this  time  although  this  will  change  next  phase.  The  recommended 
memory  is  32MB,  but  this  is  strongly  dependent  on  the  details  ( number  and  size  of 
polygons  ) of  the  environment.  We  have  32MB  on  the  220GTX.  For  medium  size 
environmerit  ( — 3000  polygons ) this  program  is  graphic  bound,  even  though  the  4D  220 
GTX  is  capable  of  100,000  ( 10  pixel,  lighted,  Gouraud  shaded  triangles)  polygons/sec. 
The  problem  is  exacerbated  as  more  windows  are  opened  because  the  entire  scene  must  be 
drawn  multiple  times. 


While  these  workstations  provides  the  performance  we  require  for  the  integrated  VEJI  VP 
JACK  and  LM  can  be  run  independently  on  less  costly  workstations.  An  earlier  version  of  * 
VF  ran  on  an  Amiga.  A newer  version  is  under  development  for  the  SGI  Personal  Iris  A 
version  of  LM  runs  on  the  Sun  family  of  workstation  and  has  no  graphic  requirements 


4.3  SOFTWARE  ENVIRONMENT 

The  entire  visibility  software  modules  have  about  100,000  lines  of  code  and  are  completely 
written  m the  C programming  language.  It  is  comprised  of  three  major  software 
components. 


The  first,  VP , which  stands  for  Volume  Perimetry,  is  the  portion  of  the  software  that 
represents  the  Volume  Visual  Field  (VVF ) and  retinal  images  as  distinct  graphical 
constructs  in  separate  windows  that  are  yoked  to  the  fixation  point.  VP  illustrates  the 
effects  of  changes  in  the  fixation  point  with  respect  to  the  WF  or  object  image  projections 
onto  the  observer  s retina.  This  software  is  located  in  the  directory  of  J 

$UPENN/gen/src/jack/vp_src  and  was  supplied  under  a contract  from  NASA-Ames  A3I 
by  Aries  Arditi  and  Steve  Azueta  from  the  Lighthouse  Inc.  Research  Labs. 

The  second  software  component,  the  Legibility  Model  ( LM ),  computes  the  probability  for 
correct  discrimination  between  two  symbols  based  on  stimulus  characteristics,  7 
environmental  factors  and  observer  state.  This  software  was  produced  at  SRI/David 
Samoff  Research  Center  by  Jeff  Lubin  under  contract  from  NASA-Ames  A3I.  Due  to  the 
intensive  computational  requirements  of  the  model,  and  the  desirability  of  building  an 
interactive  visibility  tool,  legibility  data  is  precomputed  and  stored  in  files  LM  is  a 

»le‘cly.stand-al0?c  Pr°Sram  and  only  data  files  are  access  during  the  running  of 
VEJI.  During  operation  of  the  VEJI,  based  on  the  current  vision  relevant  parameters  the 
appropriate  data  files  are  read  in  and  displayed.  Currently  the  data  files  must  be  located  in 
the  same  directory  that  the  executable  called  from.  This  restriction  will  be  changed  in  a 
luiiirc  release. 
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The  third  software  component  is  Jack,  the  anthropometric  modeling  fwl^ssoftware 
provides  the  kinematically  correct  models  of  stereotypical 

software  was  produced  at  the  University  of  Pennsylvania  by  Norman  Badler  and  Cary 
Phillips  under  a partial  grant  from  NASA-Ames  A I. 

4 4 EXTERNAL  INTERFACE  REQUIREMENTS 

described  in  the  standard  Jack  documentation. 

There  are  no  network  interfaces  for  the  vision  model.  However,  future  plans  may 

for  further  details. 

5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

5.1.1  Design  Approach  and  Tradeoffs 

The  design  of  a vision  model  that  could  predict  the  suffic^ 

rlrnnvo  the  n Qtimulus  is  beyond  the  current  capabilities  of  the  MIDAS  simulatton.  mere 
STSL SS  Option  can  be  influenced  by  facers  such  as  nogn.uvelMd.ng 

r$s? s 7i ’’sssss^im  pom, 

nossibleAmpossible  to  perceive  the  stimulus  within  a given  probability.  The  firet  desig 
tradeoff  wasto  settle  on  a rather  modest  set  of  visual  properties  to  model  inorder  to 
produce  performance  probability  contours.  Still,  many  design  factors  can  be  eva  u t 
using  just  what  we  know  about  these  early  vision  properties. 

A second  tradeoff  was  to  keep  the  system  interactive.  Design  is  such  an  iterative  process 
that  non-interactive  design  tools  can  increase  design  time  significantly. 

5. 1.1.1  Legibility  Model 

It  has  been  shown  [ Ginsburg  84  ] that  contrast  sensitivity  is  a more  accurate  predictor  of 
to  thesundani  Snellen  type  visual  acuity  « u Fon flm. rMsouche 
legibility  model  is  sensitive  to  changes  in  contrast  and  is  similar  to  Watson  cortex  mod 
[Watson  85  ] with  some  notable  extensions. 

Due  to  the  computational  demands  of  the  model,  the  time  required  to  compute  the  results 
2£S2S«  Sues  sets)  takes  between  1 and  2 minutes  on  our  fastest  machine  a 4D 
220  GTX  Since  it  was  important  to  keep  the  system  responsive  to  the  user  these  data  sets 

run  time  the  state  parameters  am  constrained  sothey  may 
only  be  set  tomatoes  that  have  a data  set  associated  with  them/ The .legibility  model  is  still 
under  development  and  has  not  been  optimized.  An  initial  look  at  die  kgibihty 'model 
suggests  that  it  could  be  optimized  to  compute  a data  set  in  under  20  seconds  on  a 4D  220 

GTX. 
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The  legibility  data  is  projected  with  the  same  functions  that  are  used  to  project  retina  objects 
nmhSm  ^™r°nment:  This  was  the  simplest  way  to  present  the  data  but  leads  to  a J 
proWem.  The  data  is  only  valid  for  the  MFD  at  a fixed  distance  from  the  observer,  using 
NffD  character  descriptions  and  contrast  information.  It  therefore  makes  no  sense  to  place 

1 T **  Surface  of  lhe  MFD-  Unfortunately,  there  is  nothing  to 
stc  p the  user  from  projecting  the  contours  anywhere  in  the  crewstation,  perhaps 
investigating  the  legibility  of  labels  on  the  panel.  This  just  isn’t  coirect  and  further  it  not 
even  possible  to  predict  how  incorrect  it  maybe.  The  user  should  never  try  to  use  these 
contours  on  anything  besides  the  MFD!  It  is  my  contention  that  the  tool  should  not  allow 
this  data  to  be  placed  where  it  is  not  valid  but  at  this  time  there  are  no  restrictions. 

5, 1.1. 2 View  Cones 

View  cones  were  selected  as  a natural  way  to  show  volumes  with  a given  angular  size  The 

S ,a£PH°aCh  *nvolvedcreating  apsurf  formatted  viewcone  for  a specific  environment  that 
as  the  desired  size  in  both  steradians  and  length.  They  were  made  with  a specific  color 
according  to  which  eye  they  would  be  projected  from.  They  were  also  made  semi- 
transparent so  that  the  user  could  see  the  intersections  of  the  cones  and  the  surface  of  the 
object  of  interest.  It  was  thought  that  a library  of  these  cones  could  be  made  and  the  user 
would  just  have  to  select  the  desired  one. 

There  were  two  problems  with  this  approach.  The  first  was  that  the  drawing  routine  for  the 

? rT  iha<!  l°  jknOW  ‘hat  the  view  cones  were  special  psurf  objects  ( they  are  on 
the  object  link  list)  and  not  to  draw  them  in  the  retina  window.  This  required  having  a 

special  check  every  time  it  was  to  draw  an  object  in  the  retina  window.  What  was  worse 
was  that  everyume  a new  cone  was  added,  the  draw  routine  had  to  be  modified  to  look  out 
for  that  cone.  The  second  problem  was  in  the  number  of  different  cones  that  would  be 
^eded,in,area  application.  There  are  an  infinite  number  of  different  size  cones  that  might 
be  needed  given  the  specifics  of  a task  domain.  Even  with  a large  library  of  cones  it  was 
likely  that  the  user  would  not  find  the  one  he  really  wanted. 

t0  the.Pr0bl.e?1  Pseud°-°bjects  were  introduced  in  Jack.  These  are  dynamic 
objects  that  are  not  seen  ( they  are  not  on  the  object  link  list)  by  the  normal  Jack  window 
draw  routines.  They  are  not  psurf,  but  are  vertices  that  are  computed,  in  real  time  when  a 

know  al^Ctt  th°ne  Ih  thlS  CaSC  * is  created  or  moved.  Since  the  Jack  draw  routines  don't 
know  about  these  objects  it  is  not  necessary  to  explicitly  check  for  them  in  retina  window 

fen^hnf  ,heS'  er’  ? "T  Possible  at  ™ dme  to  set  the  angular  size  of  the  cones  The 
hli  kH  1S  SCii  l°  .the,flxatl0n  distance  with  the  apex  origin  at  the  nodal  point  of 
the  eye.  It  is  dynamically  sized  and  positioned  as  the  fixation  point  is  varied.  This  is 
consistent  with  the  way  in  which  the  other  vision  features  work  and  also  allows  for 
arbitrary  sized  cones. 

rh^k  ,Ch°^niCOU|ld  be  proJfCted  fr°mLanywhere  not  just  the  figure’s  eyes.  At  this  point  the 
thS'hL0^:^°1Ce  ,mpement,ed’  but  llI!?a>'be  desired  to  project  them  from  a define  site 
as  the. des  gn  eye  or  sens°E  location.  This  would  change  the  implementation  of  how 
cones  are  done  now  because  a fixation  point  associated  with  the  cones  could  not  be 
assumed. 

5. 1.1. 3 Color 

lT8hr^  r°n  T^el  thf  color  rcd  is  associated  with  the  right  eye  and  the  color 
green  with  the  left  eye.  Where  they  both  mix  ( binocular ) the  color  yellow  is  used  It  is 
important  to  maintain  color  consistency  through  the  model  but  there  have  been  some 
undesirable  effects  caused  by  trying  to  maintain  this  approach.  First,  the  technique  of  using 
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ainha  blending  to  achieve  transparency  and  color  blending  allows  the  model  to  support 
Sese  features  in  real-time.  In  general  it  produces  good  results,  but  the  view  angle  can  be 
nlaced  in  such  a way  that  the  correct  intersection  of  the  left  and  nght  view  cones  is 
presented  correctly  Specifically,  alpha  blending  occurs  anytime  two  transparent  ^lygons 
Sd^thatover  each  other  on  the  screen  but  intersection  real  only  occurs  if  the  depth  of 
Z Sws  overia? als? Sbice  the  distance  between  the  cones  is  never  very  large  relative 
m thd/sTre  the  errors  never  very  large.  In  addition  it  is  only  erroneous  at  certain  viewing 
angled tmdhther^ore  is  not  a big  issued  this  time.  However,  the  user  should  be  aware  of 

this  inaccuracy. 

A second  design  issue  that  involves  color  occurs  when  retina  objects  are  jffesented  on  the 

the  display.  You  cau  still  go  back  to  the  line  mode  and  see  all  of  the  data 

contours. 

5.1. 1.4  Mapping  Inaccuracies 

The  retina  window  and  the  Aitoff  charts  both  plot  the  vertices  of  the  data  correctly  but  then 
draw  straight  lines  to  connect  these  vertices.  In  Jack's  environment  this  is  correct  but  on 
nolar  mans  straight  lines  don’t  always  map  to  straight  lines.  This  results  in  an  inaccuracy 
{hat  is  proportional  to  the  length  and  orientation  of  the  line.  The  lines  could  be  ^eipolated, 
something  like  how  circles  are  represented,  but  this  would  push  response  time  for  all  b 
the  simplest  environments  out  of  the  interactive  range.  It  seems  like  a good  tradeoff  could 
£ £g by Sg  With  the  mapping  function  the  way  it  is  now  but  when. accuracy ! £ 
important  the  more  time  consuming  but  accurate  mapping  fanction  could  be  used. /This 
funS  has  not  been  implemented  at  this  time.  The  new  4D  VGX  machine  can  now  do 
texture  mapping  in  real-time  and  would  be  the  most  accurate  and  fastest  way  do  address 

this  problem. 
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Figure  2.  MIDAS  Vision  Model  Architecture 
5.1.2  Architectural  Design  Description 

Figure  2 shows  a high  level  diagram  of  the  vision  model’s  architecture.  First  note  that  the 
boxes  with  solid  dark  drop  shadows  represents  data,  boxes  with  light  drop  shadows  *" 
present  processes  that  generate  or  use  the  data.  Starting  with  the  Legibility  model,  note 
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that  it  takes  as  inDut.  stimulus  data  and  a pixel  description  of  the  MFD  character  or 

Am.  arc  .K,  direct  Itafa  with  there*,  of  the  ™o" 
model.  This  is  the  most  porable  module  of  the  vision  model.  The  rest  of  the  vision  mode 
then  makes  use  of  the  geometrical  data  produce  by  a CAD  system,  in  this  case  MuldGen 
to  display  an  environment.  VP  is  integrated  to  Jack  via  VEJ1.  VP  makes  use  of  thedata  sets 
by  reading  them  into  the  retina  editor.  The  data  is  displayed  in  the  editor  window  and 
desired  then  projected  back  out  into  environment. 

5.1.3  External  Interface  Design 

VEJI  is  an  easy  to  use,  mouse  driven,  multi-windowed  program.  The  anthropometric 
model  in  Jack  was  extended  to  include  eye  coordinates  and  fixation  point  The  normal  jack 
interface  ( Badler  1989 ) was  extended  to  include  the  ability  to  manipulate  vision  relevant 
narameters  The  vision  options  are  included  in  the  normal  Jack  menu  space,  and  are 
KS  £ thTo$,ns  portion  of  the  menu.  S«  secdon  6.1.4.2.1  of  *5  S“S; ‘ 
documentation  for  a complete  explanation  of  the  extended  menu  space.  Under  the  options 
menu  pullout,  the  vision  relevant  selections  are:  Retina  Display,  Retina  Editor,  i ^ 
Information.  The  designer  can  interactively  set  the  fixation  point,  via  the  mouse,  anywhere 
in  3D  space  as  long  as  it  is  within  the  normal  eye  joint  limits.  The  fixation  point  can  also 
set  to  predefined  points  (Jack  sites ) . As  the  fixation  point  is  vaned  the  designer  can  watch 
the  vision  retina  plots  and  the  field  of  view  cones  reflect  those  changes  in  real  time.  The 
usercan  set  model  parameters,  such  as  ambient  light,  font  size,  etc.,  via  slider  bars. 

The  user  must  be  able  to  relate  dynamic  vision  characteristics  to  the  objects  withm  the 
design  environment  and  make  design  decisions  based  on  those  relationships.  Data 
visualization  techniques  are  employed  to  illuminate  these  relations  and  are  discussed  further 
in  the  following  sections. 

5.2  DETAILED  DESIGN 

5.2.1  Detailed  Design  Approach  and  Tradeoffs 

5.2.1. 1 Volume  Perimetry  Considerations 

Of  primary  concern  to  the  designer  is  that  portion  of  the  environment  that  falls i withm  the 
aviator's  \asual  field.  Volume  perimetry  ( a generalization  of  visual  ftelds  ) cm  te  viewed 
as  a way  of  studying  specific  geometric  relationships  between  objects  in  the  world  and 
objects  on  the  retinas.  Several  key  concepts  enhance  this  goal. 

5.2.1.1.1  Three  Dimensionality  of  Retinal  Images 

While  each  of  the  retinal  images  is  considered  to  be  two-dimensional  with,  for  example, 
horizontal  (x)  and  vertical  (y)  coordinates,  the  composite  of  the  two  retinal  images  is 
viewed  as  three  dimensional,  with  an  additional  coordinate  denoting  the  horizontal 
difference  or  retinal  disparity  between  the  image  position  of  a world  object  point  in  the  two 
eves  or  equivalently,  the  amount  of  rotation  of  one  or  both  eyes  required  to  bring  the 
fmages  of  a world  object  point  into  registration  . This,  of  course,  is  one  way  the  visual 
system  extracts  depth  coordinates  in  biological  vision. 

5.2. 1.1.2  Concurrent  Retinal  Images 

Retinal  information  and  information  about  the  visual  world  that  is  imaged  on  the  retinas 
should  be  viewed  concurrently.  In  this  way,  the  impact  of  transformations  in  one  domain 
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can  be  appreciated  in  the  other.  Motion  of  world  objects,  for  example,  will  result  in  retinal 
image  motion,  but  often  in  non-intuitive  directions  and  speeds — viewing  a yoked  display  of 
visual  space  and  retinal  space  side  by  side  can  be  a powerful  aid  to  the  understanding  of 
such  motion.  Similarly,  changes  in  global  and  local  retinal  sensitivity  over  time  can 
drastically  alter  the  visibility  of  objects  in  space.  The  ability  to  view  a rendition  of  visual 
space  that  indicates  visibility  of  objects  would  be  of  obvious  value. 

5.2. 1.1.3  Retinal  Projections 

Retinal  images  are  projections  of  visual  world  objects  onto  the  retinas.  A kind  of  converse 
operation  that  we  call  here  retrojection  is  the  construction  of  the  locus  of  possible  points  in 
the  visual  world  which  may  give  rise,  through  projection,  to  a specified  point  on  the 
retma(s).  Projection  and  retrojection  are  the  operations  which  relate  visual  world  geometry 
to  retinal  geometry.  7 

5.2.1. 1.4  Volume  Visual  Field 

The  volume  visual  field  (V VF)  is  the  locus  of  points  in  the  visual  world  which  fall  on 
sensitive  retina.  Since  retinal  sensitivity  varies  over  space  and  time,  and  since  the  size  and 
shape  of  the  VVF  changes  with  eye  position,  it  should  be  viewed  as  a dynamic  construct. 

5.2.1. 2 Legibility  Model  Considerations 

Sarnoff  s task  in  the  A3I  project  is  to  develop  a quantitative,  easily  computable  model  of 
instrument  visibility  within  an  aircraft  cockpit  environment.  The  purpose  of  the  model  is  to 
provide  visibility  data  to  display  designers,  who  need  quantitative  information  on  the  effect 
that  their  design  choices  will  have  on  crew  performance  over  a broad  range  of  mission 
scenarios.  Given  this  range,  the  visibility  of  alphanumeric  and  other  information  depends 
not  only  on  the  spatial  configuration  of  the  display,  but  also  importantly  on  lighting 
parameters  such  as  light  level  in  the  cockpit  and  the  observer’s  state  of  adaptation  The 
model  must  therefore  accurately  assess  the  effect  of  such  parameters,  and  convey  this 
information  to  the  display  designer  in  an  easily  understandable  form. 
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Legibility  Model  Overview 
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Figure  3.  l egibility  Model  Overview 


A concemual  diagram  of  the  legibility  model  is  shown  in  Figure  3.  The  inputs  to  the  model 
^ a S' KX  environmental; "and  observer  variables.  Tire  output  of  the ^model  are  a 
narametric  data  set  that  spans  the  parameter  ranges  of  interest.  A more  detail  description 
will  be  given  in  section  52.2.2  under  detailed  design  of  the  Legibility  model.  For  a more 
complete  description  of  the  legibility  model  see  appendix  B. 

5.2. 1.2.1  Basic  strategy 

Samoff s approach  to  the  NASA  legibility  modeling  task  has  been  to  augment  the  one- 
dimensional  discrimination  models  of  Carlson,  Barten,  and  the  basic  psychophysics 
community  to  include  the  twodimensional  spatial  processing  used  in  the  Watson  detection 
mS  In  addition,  because  the  NASA  task  requites  performance^^ 
ranee  of  lighting  conditions  and  observer  perceptual  states,  the  Sarnoff  model  includes  a 
Sh  Ii  nre  nrocesses  the  input  images  to  model  the  effects  of  changes  in 
mlfnlS rSSSS  3 observer  fixation  locarion  iu  ihree-tonrional  space. 

5.2. 1.2.2  Choice  of  performance  measure 

The  performance  measure  chosen  for  the  Samoff  model  is  probability  of  a correct 
discnmination  between  two  input  images;  e.g.,  between  two  alpha-numenc  characters. 
Other  performance  measures  are  of  course  possible;  one  common  measure  from  the 
appliedpsychophysics  community  is  image  quality,  expressed  in units  of  just-noticeabe 
differences  (JNDs)  between  the  two  images.  In  fact,  the  probability  measure  used  in  the 
Sarnoff  model  is  derived  from  a JND-like  measure,  as  will  be  described  below.  Another 
potentially  useful  measure  is  the  reaction  time  required  for  correct  discnmination. 
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However,  this  measure  has  received  little  attention  to  date  in  psychophysics  research,  and 
its  use  here  would  thus  require  a large  amount  of  additional  data  collection  and  modeling. 
Another  important  consideration  in  the  formulation  of  the  Samoff  model  is  that  the 
performance  measure  be  conservative.  That  is,  display  design  engineers  need  a ma«uw 

25iriSSiiS?W  ™le  °Ul  VC? bad  desi8ns>  but  would  letmarginal  designs  pass  for 
testing.  In  other  words,  it  is  better  to  err  on  the  side  of  overestimatingrather 
than  underestimating  the  probability  of  successful  discrimination. 

5.2,1. 2. 3 Incremental  improvement 

fn  "10del  baf  constructed  in  such  a way  as  to  allow  incremental  improvement 

m its  ability  to  predict  performance  among  complex  stimuli.  For  example,  the  model  as 

formulated  can  accurately  predict  performance  only  among  static,  monochromatic 
images.  However,  by  replacing  the  model  s spatial  filters  with  a set  of  spatio-temporal 
tilters,  performance  measurement  among  moving  stimuli  would  be  possible.  The  strategy 
in  model  development  is  therefore  to  validate  the  simple  model  on  simple  stimuli,  and  then 
incrementally  build  in  model  complexity  to  handle  the  more  complex  stimuli. 
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Binocular  Uision 
Hierarchy  Chart 


Figure  4.  Binocular  Vision  Hierarchy  Chart 


5.2.2  Detailed  Design  Description 
5.2.2.1  Compilation  Unit 

The  software  implementation  of  the  vision  model  can  be  thought  of  as  being  comprised  of 
four  main  groups  (see  figure  4): 

1.  The  initialization  and  support  functions;  which  is  the  group  of  functions  that  perform 
general  house  keeping  activities. 

2.  The  Eye  model  editor,  the  group  of  functions  that  allow  editing  the  definition  of  eye  and 
environmental  parameters. 

3.  The  Display  Manager,  the  group  of  functions  that  maintain  the  various  window  displays. 

4.  The  Models;  the  group  of  functions  that  model  stimuli,  the  environment,  and  the 
observer. 
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Figure  5.  Initialization  and  Support  Functions 


5. 2. 2. 1.1  Initialization  and  Support  Functions. 

As  shown  in  Figure  5,  the  initialization  and  support  module  is  comprised  of  submodules 
used  to  initialize  the  vision  system  and  by  modules  to  perform  mundane  activities  such  as 
user  input  and  vector  arithmetic. 

5. 2. 2. 1.1.1  Time  Base  ( not  implemented  ) 

"Hiis  module  provides  a tick-based  approach  to  time  simulation.  Many  vision  responses  are 
time  variant  and  require  the  maintenances  of  a time  function.  These  functions  are  not 
implemented  at  this  time. 

5.2. 2. 1.1. 2 Vector  Functions 

pis  module  provides  some  basic  vector  arithmetic  functions.  There  is  a common  need  to 
define  vector  and  perform  arithmetic  on  them.  This  module  defines  the  data  structure  for  a 
vector  and  provides  the  functions  such  as  dot  and  cross  products,  vector  adds  and  subtract 
etc.  These  function  can  be  found  in  the  file  vector. c 

5.2.2. 1.1. 3 Initialize  Vision  Model 
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This  module  provides  the  basic  functions  to 
Also  a configuration  file  must  be  read  to  set 
function  can  be  found  in  the  file  sctstatc.c 


read  and  initialize  the  environmental  database, 
the  environment  to  a user  defined  state.  These 


5. 2. 2.1. 1.4  I/O  Functions 

the  file  retina Jight  for  data  files  for  the  retina  editor. 
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Figure  6.  Eye  Model  Editor 


5.2.2. 1.2  Eye  Model  Editor 

The  philosophy  of  the  vision  model  design  is  to  allow  the  user  to  modify  any  Dortion  of 
the  vision  model.  It  is  the  nature  of  design  to  consider  "what  if'  type  quLtionsand  the 
vision  model  must  be  sufficiently  flexible  to  allow  a large  variety  of  questions  Some  of  the 
editing  could  be  performed  by  a designer.  Other  parts  of  the  editing  would  be  used  by 
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human  factor  engineers  to  tune  the  model  to  their  application.  A set  of  5 editors  will  be 
provided  to  assist  the  user  in  entering  data.  See  Figure  6 for  an  overview. 

5.2.2.1.2.1  Retinal  Model  Editor 

The  retinal  model  is  intended  to  provide  the  user  with  an  easy  method  » describe  spatial 
relationships  between  the  retinal  map  and  areas  typically  found  on  the  retina.  These 
ESS  be  found  in  the  file  renmjightx  TJe  software  has  not  progn* dal he 
where  the  distinction  between  the  objects  effects  how  they  are  projccKd  All  tto  ts  entered 
pretty  much  the  same,  either  manually  drawn  in  the  editor  or  read  from  a data 


5.2.2. 1.2.2  Perimeter  Objects 

This  class  of  objects  define  the  periphery  for  the  left  and  right  orbs. 


5. 2. 2. 1.2.3  Default  Template 

This  object  defines  the  default  ( generic ) retinal  objects.  Objects  such  as  fovea,  macula  are 
predefined  for  the  X percentile  person. 

5.2.2.1.2.4  Pathological  Objects 

This  class  of  objects  defines  areas  on  the  retina  that  are  dysfunctional.  The  user  will  enter 
coordinates  via  a mouse  or  data  file  that  describes  the  geometry  of  these  objects. 


5.2.2. 1.2.5  Static  After  Image 


This  class  of  objects  defines  a static  area  on  the  retina  that  will  be  associated  with  a intense 
light  source.  A more  robust  model  is  defined  in  the  temporal  eye  model. 


5.2.2. 1.2.6  Concentric  Objects 


This  class  of  objects  define  the  contours  that  share  a common  attribute.  The  user  creates  the 
objects  by  the  same  method  described  for  Pathological  object 

5.2.2. 1.2.7  Performance  Objects 

This  class  is  the  same  as  Concentric  of  object  except  that  the  geometry  is  not  concentric. 

5.2.2. 1.2.8  Optics  Model  Editor  ( Not  Implemented  ) 

This  editor  has  the  facilities  to  store  and  modify  optical  attributes,  such  as  myopia,  of  the 
eye  model. 


5,2.2. 1.2.9  Temporal  Model  Editor  ( Not  Implemented  ) 


This  editor  has  the  facilities  to  store  and  modify  temporal  attributes,  such  as  photoreceptor 
light  sensitivity,  of  the  eye  model.  This  model  would  be  necessary  to  track  the  adaptive 
state  of  a pilot  over  the  course  of  the  simulation. 


5.2.2.1.2.10  Geometric  Model  Editor  ( Not  Implemented  ) 

This  editor  has  the  facilities  to  store  and  modify  geometric  attributes,  such  as  nodal  points 
location,  of  the  eye  model. 
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5.2.2.1.2.11  Environmental  Factors  Editor  (Not  Implemented  ) 

This  editor  has  the  facilities  to  store  and  modify  environmental  factor  that  effect  visual 
performance,  such  as  fog  attenuation,  of  the  eye  model. 


Figure  7.  Display  Manager 


5.2.2.1.3  Display  Manager 

The  Display  Manager  coordinates  the  presentation  of  data  with  the  in  place  windowing 
system.  System  dependent  windowing  calls  are  used  by  this  module  to  display  the 
requested  data.  See  Figure  7 for  an  overview  of  t his  module.  If  this  software  is  ported  to 
other  systems  not  running  the  same  windowing  system,  this  module  will  require  special 
attention.  Most  display  functions  can  be  found  in  retina.c  and  window Jight.c 

5. 2. 2.1. 3.1  Retinal  Model  Display 

The  Retinal  Model  Display  is  a graphical  representation  of  retinal  features  drawn  in  fovea 
centric  polar  coordinates.  The  retinal  map  along  with  the  retinal  objects  defined  in  the  retinal 
editor,  are  in  a window  of  its  own.  These  objects  can  then  be  projected  out  into  the 
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Centric  Projection  shows  an  example  of  this  type  of  display. 

5.2.2. 1.3.2  Fixation  Display 

be  found  in  retina.c  and  vp  jrcl retina  Jight.. 

5.2.2. 1.3.3  Volumes  Display 

The  Volumes  Displays  can  be  an  independent  window  or  and  overlay  on  an  existing 
JSSSS™  Visual  perfomtance  data  is  best  vinjahaad* art 
area  coverage  of  the  field  of  view  is  one  such  data  set  In  appendix  A the  figure  “led  Jack 
1SS«  Window  shows  au  example  of  FOV  infonnanon  tmrlatd  on  the  envuonment. 

5.2.2.1.3.4  Function  Display  ( not  implemented  ) 

P-an^rtc 

situations  without  having  to  simulate  them. 

5.2.2.1.3.5  Environmental  Display 

The  Environmental  Display  is  used  to  display  a graphical  view  into  *c.envton!1|fH^  . 
database  One  or  more  windows  may  be  opened  into  the  database.  A six  degree  of  freedom 
SjaSlt  Say  ?e  selected.  Objects  r£y  be  defined  and  rop^rtjemec 1 m dtese  windows. 
Th^c#*  nhiecK  are  one  wav  to  provide  the  stimuli  to  the  vision  model.  The  environmental 
display  is  also  where  the  visual  performance  data  is  projected.  This  allows  repsnation 
Sew  performance  data  and  the  object  that  they  were  based  on.  In  Appendix  A the  figure 
titled  Projection  of  Legibility  Contour  shows  an  example  of  an  environmental  display  with 
FOV  (cone  projecting  from  the  eyes)  information  overlaid. 
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Figure  8.  Models 


Figure  8 shows  the  further  break  down  of  the  models  group.  The  degree  to  which  the 
models  are  implemented  varies  widely.  As  the  simulation  requirements  are  defined  for  the 
be^argetedto-' expansion  ^ f particular  models  will  become  known  and  particular  models  can 
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4.1 .1.1  Objects 
Orientation 
Material  Description 
Geometry 

etc. 

4. 1.1 .2  Environmental 
Factors 

Precipitaon 

Clouds 

Fog,  Mist,  Smog 
Darkness 

4.1 .1.3  Texture  Maps 


4. 1.2.1  Light 
Direction  Vector 
Energy  Spectrum 
Intensity 

Diffuse,  Specular,  Emissive 

Attenuation 

Position 

4. 1.2.2  Sun 

4.1. 2.3  Flares,  Lasers 
Direction  Vector 
Energy  Spectrum 
Position 

Duration  & Start  Time 
Intensity  Curve. 

4. 1.2.4  Shadows 
Umbras 
Penumbras 
Geometric  Attenuation 


4.1. 3.1  Human  Figure 
Geometry 

State 

Age 

Fatigue 

Cognitive  Loads 

4.1. 3.2  Kinematics 
Joint  Space  and  Limits 
Waypoints 

4.1. 3.3  Head 
Orientation  and  Position 
Scan  History 
Tracking  Function 


mp  10-B9  I 
4.1 


Figure  9.  Environmental  Model 
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5.2.2. 1.4  Models 


5. 2. 2. 1.4.1  Environment  Model 

be  'ha' 

5.2.2. 1.4. 1.1  Workspace  Model  ( Not  Implemented  see  future  directions  ) 

5.2.2. 1.4. 1.2  Illumination  Model  ( Not  Implemented  see  future  directions) 

5. 2. 2. 1.4. 1.3  Anthropometric  Model 

m0del,is  onIy  needed  for  vision  issu«  if  self  occlusion  issues  are  of 
merest.  Otherwise  a simpler  eye  coordinate  model  could  be  used  If  an  anthronomomhir 

to^e  eyeUm^dCn  user/simulation  can  manipulate  this  model  'and  the  data  Sid  aStoput 


Human  Figure  Geometry 

Adaptive  State  of  the  Pilot  ( not  implemented  ) 

Head  Specific  Attributes 

Orientation  and  Position 

Scan  History  ( not  implemented ) 

Tracking  Function 
Kinematics 

Joint  Space  and  Limits 
Paths 

Inverse  Kinematics 
Etc. 


See  Jack  User  Guide  for  a detailed  description. 
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Geometric 

Model 

4.2.1 


Retinal 

Model 

4.2.2 


Optical 

Model 

4.2.3 


Temporal 

Model 

4.2.4 


Figure  10.  Eye  Model 


5.2.2.1.4.2  Eye  Model 

The  eve  model  has  four  submodels  ( see  figure  10).  The  submodels  are  a natural  way  to 
group^the  data  structures  that  describe  static  features  such  as  eye  separation  °r  ph°t 
receptor  mosaic,  with  the  functions  for  computing  the  current  value  for  time  dependent 
data  such  as  pupil  size  based  on  those  data  structures. 

5.2.2. 1.4. 2.1  Geometric  Model 

This  model  describes  the  eye  in  terms  of  its  geometric  properties  and  client  po »nd 
orientation.  This  information  is  currently  spread  out  over  the  enure  i iystem. ■ ^ 
release  it  may  be  accessed  in  one  place  so  that  specification  would  be  easier  and  more 
explicit.  Some  examples  of  geometric  data  are  as  follows: 


Geometric  Attributes 

Orientation  and  Position 

Size,  Focus,  Major  and  Minor  Axis 

Interpupillary  Distance 

Pupil  Diameter 

Motion  Limits 

Perimetry  Geometry 

Limiting  Facial  Structures 

Goggles,  etc. 


Page  H-27 


5.2.2. 1.4.2. 2 Retinal  Model 


The  user  is  required  to  design  a retinal  model.  He  can  choose  for  simplicity  the  default 
model  which  is  simply  a retina  with  a fovea  and  biological  blind  spot.  If  the  user  wishes  to 
investigate  a unique  retinal  model  he  can  enter  retina  specific  data  such  as  scotomas  or 
unusual  distribution  functions.  Some  examples  of  this  kind  of  data  is: 


Distribution  Functions  for  Cones  and  Rods 

Macular  region 

Receptive  Fields 

Spectral  Sensitivity 

Rod  Sensitivity  Function 

Cone  Sensitivity  Function 

Binocular/Monocular  Map 

Retinal  Mapping  Functions 

Color  Vision 


5,2.2. 1.4. 2. 3  Optical  Model  ( not  implemented  ) 

This  model  describes  the  optical  properties  of  the  left  and  right  eyes.  The  default  data  is 
vaSSvbLfhe  - ,standard  ( aPPfndix  ),  but  again  the  user  can  enter  his  own.  Most 

H^c^bLdC  aS  not  **  evaluated  eg.  not  contribute  to  the  computation  if  so 

desired.  Mostly  this  Mode  is  not  complete.  Here  are  a few  of  the  features  you  would 
expect  to  find  in  this  model:  3 


Accommodation 
Index  of  Refractions 
Optical  Axis 
Fixation  Point 
Diopter  Range 
Convergence  Angle 
Refractive  Errors 
Aberrations 

Attenuation  such  as:  Vitreous  Fluids,  Cornea,  Lens 
Scatter  and  Glare  due  to  Vitreous  Fluids,  Cornea,  Lens 
Veiling  Glare  and  Discomfort  Glare 

5.2.2.1.4.2.4  Temporal  Model  ( not  implemented) 

This  model  would  include  functions  such  as: 

Sensitivity  to  light 
Light  Adaptation 
Temporal  Sensitivity 
Spatial  Sensitivity 
Eye  Movements 
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5.2. 2. 1.5  Directory  structure 

Figure  1 1 shows  the  directory  structure  of  Jack  as  distributed  with  the  vision  system.  See 
the  Jack  User  documentation  for  further  details. 


The  following  data  files  are  distributed  with  the  vision  model.  These  files  are  generated  by 
fcons  when  run  on  the  legibility  model  data  files.  They  are  in  a format  that  can  be  read  by 
the  retina  editor  and  should  reside  on  the  directory  you  start  up  VEJI  from. 


QO_i  1 _1 1 _d  1 .cons 
QO_j2_l  l_d  1 .cons 
QOJ3_ll_d  Leons 
QO_i4  J 1 .5_d  1 .cons 
QO_i4_l  l_d  1 .cons 


QO  J4_12.54_d  1 .cons 
QO_i4_12_d  1 .cons 
qo_iO_ll_d  Leons 
qo_il_ll_d  Leons 
qo_i2_ll_dl.cons 


qo_i3Jl_d  Leons 
qo_i4  1 .5_d  1 .con  s 
qo_i4_ll_d  Leons 
qo_i4_12.54_d  1 .cons 
qo_i4_12_d  l.cons 


The  following  makefile  is  used  to  remake  a new  version  of  cortex  the  name  of  the  legibility 
model  executable.  Use  the  command  make  in  the  directory  with  the  source  code  This  in 
most  cases  will  be  in  UPENN/legibility/src. 

Makefile: 


CFLAGS=  -g  -float 
FFLAGS=  -g 
LDFLAGS= 

LIB=  -lm 
#SGI  machines 
LIB=  -lm  -lmalloc  -lgl 
DISPLIB= 

INCLUDE=  imgdec.h  imgmacro.h 
EXECUTABLES=  cortex 

IMGLIBSOURCES=  imgio.c  imgalloc.c  imgconv.c  arralloc.c  misc.c  flmgimg.c  reflect  c 
redexp.c  fimgopmike.c  filtsub.c  dispimg.o  kernel  .c 


IMGLIBOBJ=  $(IMGLIBSOURCES:.c=.o) 

all:  $(EXECUTABLES)  imglib.a 

imglib.a:  $(IMGLIBOBJ) 

arrv  $@  $(IMGLIBOBJ) 
ranlib  $@ 

$(EXECUTABLES):  $$<a>.o  imglib.a 

$(CC)  -o  $@  $@.o  imglib.a  $(CFLAGS)  $(LIB) 

$(IMGLIBOBJ):  $(INCLUDE) 
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The  following  is  a listing  of  all  the  files  that  make  up  cortex  ( legibility  model  executable ). 
This  in  most  cases  they  will  be  in  UPENN/legibility/src. 


arr  alloc  .c 

btest.c 

corfish.c 

cortex.c 

dispimg.c 


fcons.c 

filtsub.c 

fimgimg.c 

fimgop.c 

fimgopmike.c 


imgalloc.c 

imgconv.c 

imgdec.h 

imgio.c 

imgmacro.h 


kemel.c 
malloc.c 
misc.c 
redexp.c 
re  fleet. c 


The  following  makefile  is  used  to  rebuild  a new  version  of  jack.  The  executable  will  be 
named  djack. 

Makefile: 


NAME  = djack 
VERSION  = 4.6 

FULLNAME  = $(NAME)-$(VERSION) 

BINDIR  = ,./../../4D/bin 
EXE  = $ (FULLNAME) 

OBJ  = menu.o  retina_menu.o  retina.o  options.o  vision_info_menu.o\ 
editor_menu.o  peawin.o  ret_editor.o  setstate.o  viewcones.o  body.o 
SRC  = $(OBJ:.o=.c)  Makefile  _ 

LIBS  = -L$(LIBDIR)  -ljmenu  -Igrace  -ljcmds  -ljack  -lpea  -lpsurt  \ 

-lalt  -lgio  -lvec  -lrle  -leditor  -lgl_s  -lsun  -lbsd  -lmalloc  -lm  -lc_s 


all:  $(EXE) 

^C^ffH^DFLAGS)  $(CFLAGS)  $<OB„  SIMM) 

$(OBJ) : $(INCLUDEDIR)/jack.h 
include  $(INCLUDEDIR)/make.h 


The  following  files  are  modified  Jack  files  or  new  additions  to  Jack.  They  are  the  files  used 
in  the  above  makefile  to  build  the  new  executable.  The  new  files  are  printed  in  bold  letters. 
If  any  Jack  source  file  was  modified  the  modification  was  delineated  by  comments  with  the 
key  words  A3I  in  them.  The  modified  files  were  removed  from  their  normal  Jack 
directories  and  placed  here. 


body.c 

editor  menu.c 
ret  edltor.c 
retma.c 
menu.c 


options. c 
pea  win. c 
viewcones.c 
vision_info_menu.c 
retina_menu.c 


retina_mod.c 
setstate.c 
info  menu.c 
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The  following  makefile  builds  a new  VP  library  that  is  used  to  link  with  Jack  Issue  the 
command  make  in  the  directory  SUPENN/4.5/gen/src/jackAp_src  and  a new  lib^ywiU 


Makefile: 

SHELL=/bin/sh 

LIB=  ../../../../4D/lib/libeditor.a 
ARFLAGS  = rvs 

OBJ  =\ 

vector.o  \ 

mouse.oX 

list_light.o\ 

draw.oX 

retina_light.o\ 

init.o  \ 

eye.o\ 

message.o  \ 

view.o  \ 

window_light.o 

SRC  = $(OBJ:.o=.c)  Makefile 

all:  $(LIB) 

$(LIB):  $(OBJ) 

ar  $(ARFLAGS)  $(LIB)  $? 

ranlib:  $(LIB) 
ar  ts  $(LIB) 

These  files  make  up  the  VP  library. 
Makefile  draw.c  eye.c 

cye.h  init.c  listjightx 

memory.c  message.c  mouse.c 

retina Jight.c  test.obj  vec.h 

vector.c  window_light.c 


Page  H-32 


Page  H-33 


5.2. 2. 2  Detailed  Design  of  Compilation  Units 

5.2.2.2.1  The  Legibility  Model 

A more  detailed  overview  of  the  legibility  that  will  facilitate  understanding  the  following 
discussion  is  provided  above  in  Figure  12. 

5.2.2.2.1.1  Input  parameters  and  stimulus  format 

The  model,  as  currently  formulated,  takes  as  input  one  or  two  image  files,  and  a number  of 
optional  lighting  and  observer  state  parameters.  These  parameters  are  listed  here  with  the 
default  value  and  units  in  parentheses: 

Screen  luminance  (10.0  foot-lamberts) 

Illuminance  (0.0  foot-candles) 

Eccentricity  of  displayed  stimulus  (0  degrees) 

Fixation  depth  (741.12  mm) 

Stimulus  depth  (741.12  mm) 


The  image  files  start  with  two  four-byte  integers  indicating  the  width  and  height  of  the 
image  in  pixels,  followed  by  rows  of  pixel  values  in  floats.  The  images  can  be  any  size 
although  should  be  at  least  256x256  to  allow  filtering  within  a large  enough  range  of 
different  frequency  bands.  The  conversion  factor  from  pixels  to  mm  is  currently 
fixed  in  the  software  as  13.21  pix/mm.  With  only  one  image  as  input,  the  software 
calculates  the  probability  of  detecting  that  stimulus;  with  two  images,  the  standard 
discrimination  probability  is  calculated. 


^n.  tiie  current  version  of  the  model,  pixel  values  are  required  to  range  from  -1 .0  to  1.0 
with  the  maximum  absolute  value  indicating  the  contrast  of  the  stimulus.  For  example*  a 
sine  grating  stimulus  with  peaks  at  0.5  would  have  a contrast  of  0.5.  To  remove  the  ’ 
need  for  this  convention,  a model  stage  which  computes  contrast  from  arbitrarily  scaled 
input  images  is  needed,  but  has  not  yet  been  implemented. 

5.2.2.2.1.2  Front  end  calculations 


Several  initial  transfomiations  on  the  input  images  are  performed  prior  to  the  linear  filtering 
stage  of  the  model.  These  transformations  model  the  effect  of  changes  in  fixation  depth 
veiling  luminance,  and  fixation  eccentricity.  v ' 


5.2.2.2.1.3  Fixation  depth 


In  order  to  account  for  changes  in  effective  image  resolution  with  changes  in  the  difference 
between  image  depth  and  fixation  depth,  we  used  geometrical  optics  to  calculate  the  size  of 
the  blur  circle,  and  then  pre-filtered  each  input  image  with  this  disk-shaped  convolution 
kernel.  This  calculation  requires  knowledge  of  the  distance  from  the  exit  pupil  to  the 
imaging  surface  (i.e.,  the  retina),  which  we  took  as  20.3  mm  from  Westheimer  (1986).  It 
also  requires  an  estimate  of  pupil  size.  For  this,  we  wrote  a simple  interpolation  routine  to 
estimate  pupil  diameter  at  any  light  level  from  a table  published  in  Hood  and  Finkelstein 
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5.2.2.2.1.4  Contrast  reduction 


Veiling  luminance,  caused  by  the  reflection  of  ambient  light  by  the  display  screen  reduces 
the  effective  contrast  of  displayed  information.  We  are  modeling  the  screen  face  a 
perfectly  lambertian  surface  with  a reflectivity  of  1CK%.  This  assumption  implies  diatan 
illuminance  of  10  fed  will  result  in  a veiling  luminance  of  1 fL.  We  are  defining  contrast 


as: 


c = (lmax  - lmin)/(lmax  + lmin) 

where  lmax  and  lmin  are  the  maximum  and  minimum  displayed  luminances.  Given  this 
definition,  the  addition  of  a veiling  luminance  v to  both  lmax  and  lmin  changes  the  contrast 
by  a factor  l/2v. 

5. 2. 2. 2. 1.5  Eccentricity  scaling 

Abundant  psychophysical  evidence  shows  that  contrast  sensitivity  remains  roughly 
constant  across  the  visual  field,  if  the  grating  patch  is  scaled  up  by  a linear function .of 
eccentricity.  This  strongly  suggests  that  processing  is  similar  across 
for  a linear  scaling  up  of  sensor  size  towards  the  penphery.  In  order  to  model  the ^effect  of 
this  change  in  sensor  size,  we  found  it  more  convenient  to  scale  down  die  size  of  die  in p 
images  as  a function  of  eccentricity,  rather  than  to  scale  up  the  size  of  the  sensors. 
used  the  scale  factor  k of  0.4  quoted  by  Watson  (1983),  so  that  the  scaling  of  input  images 
as  a function  of  eccentricity  e is 

e = 1.0/(1.0  +ke). 

5. 2.2. 2. 1.6  Linear  filtering 

5. 2. 2.2. 1.6.1  Pyramid  decomposition 

In  order  to  filter  within  a range  of  different  frequency  channels,  the  input  image  is  first 
decomposed  with  a gaussian  pyramid  into  channels  separated  from  each  other  dy 
octave^  The  frequencies  we  chose  for  these  channels  are  identical  to  those  used  by  Watson 
(1983);  i.e.,  32  through  0.5  c/d,  corresponding  to  seven  octaves  or  equivalently,  seven 

pyramid  levels. 

5.2. 2. 2*1. 6. 2 Computing  filter  gains 

The  human  visual  system  is  not  equally  sensitive  at  all  frequencies  A plot  of  contrast 
detection  threshold  as  a function  of  spatial  frequency  shows  roughly  an  inverted-U  shape, 
with  a peak  at  roughly  2 c/d,  and  complete  loss  of  sensitivity  by  approximately  60  c/d. 
Moreover,  as  shown  by  van  Nes  and  Bouman  (1967),  the  shape  of  this  contrast  sensitivity 
function  changes  with  retinal  illuminance.  To  model  these  dependencies,  the  image 
component  in  each  frequency  channel  is  weighted  by  a gain  factor  appropriate  for  the  retinal 
illuminance,  before  any  oriented  filtering  is  performed. 

Retinal  illuminance  (in  photopic  trolands)  is  calculated  as  the  amount  of  light  incident  on  the 
cornea  (in  cd/m2)  times  the  pupil  area  (in  mm2),  where  the  light  incident  on  the  cornea  is 
assumed  to  be  the  screen  luminance  plus  the  veiling  luminance,  appropriately  converted 
from  fL  to  cd/m2.  The  gain  at  each  frequency  is  then  calculated  directly  from  the  van  Nes 
and  Bouman  data  with  a simple  log  interpolation  function  to  return : sensitivities  at  retinal 
illuminances  other  than  those  reported  in  the  data.  For  example,  if  the  threshold 
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modulation  for  a 1 c/d  grating  were  1%  at  10  tds  and  5%  at  1 tds,  then  we  interpolate  the 

distance  from  .1  to  10)  to  be  2.23%  (half  the  log  distance 
from  1%  to  5%),  This  direct  calculation  is  possible  only  under  the  assumption  of  no 
summation  among  different  frequency  channels;  any  assumed  summation  would  require  a 
^^complicated  relationship  between  the  contrast  sensitivity  function  and  the  gain  of  each 


5.2.2.2. 1.6.3  Steerable  filtering 

For  convenience  and  speed  of  oriented  filter  operation,  we  use  the  steerable  filters  of 
Freeman  and  Adelson  (1990),  which  allow  separable  calculation  of  linear  filter  responses  at 
any  orientation  and  phase.  The  filters  implemented  here,  a second  derivative  of  a gaussian 
and  its  Hilbert  transform,  have  a log  bandwidth  at  half  height  of  approximately  0.7  octaves. 
This  is  within  the  range  of  bandwidths  inferred  psychophysically  (c.e.,  Watson  and 
Robson  1981)  We  are  using  four  orientations  (0, 45, 90,  and  135  degrees)  and  two 
phases  (sine  and  cosine),  for  a total  of  eight  oriented  filter  responses  per  pyramid  level. 

The  orientation  bandwidth  of  these  filters  (i.e.,  the  range  of  angles  over  which  the  filter 
output  is  greater  than  one  half  the  maximum)  is  approximately  65  degrees.  This  figure  is 
n'nooly  “PI  th®5ldegrec  tuninS  of  monkey  simple  cells  reported  by  Devalois  et  al 
(1984)  and  thC  30 1°  60  degree  rangc  reP°rte<1  psychophysically  by  Phillips  and  Wilson 

5.2.2.2.1.6.4  Energy  calculation 

During  the  early  stages  of  model  testing,  we  found  that  the  detectability  of  a simple  edge 
could  change  dramatically  with  small  changes  in  edge  position.  To  combat  this  problem,  a 
small  amount  of  spatial  summation  was  added  by  computing  energy  after  the  linear  filtering 
stage.  That  is,  corresponding  sine  and  cosine  filter  responses  were  combined  as: 

e(xi)  = sin2(x;)  + cos2(x;) 

where  Xj  is  a linear  filter  response,  indexed  over  filter  position,  orientation,  and  frequency 


S.2.2.2. 1.6.5  Point  nonlinearity 


Nachmias  and  Sansbury  (1974)  showed  that  the  results  of  a grating  contrast  discrimination 
experiment,  when  plotted  with  threshold  contrast  increment  as  a function  of  the  base 
contrastfrom  which  the  increment  threshold  is  being  measured,  produce  a dipper-shaped 
curve.  These  authors  argued  that  the  results  can  be  modeled  by  assuming  a sigmoid  non- 
linearity following  a linear  detection  mechanism.  The  decision  mechanism  has  available  to 
it  only  the  output  of  this  non-linearity,  and  reliably  discriminates  between  inputs  of  two 
different  contrasts  when  the  difference  in  outputs  is  greater  than  some  threshold. 


To  quantitatively  model  the  dipper-shaped  contrast  discrimination 
and  Foley  (1980)  in  using  a non-linear  transducer  of  the  form: 


curve,  we  follow  Legge 


7’(Li)  = rlLiln/(Li2+J2) 

where  T is  the  non-linear  transducer  output,  L,  is  the  linear  filter  response  (indexed  as 
above),  r is  an  overall  gain-setting  parameter,  n is  a real  number  greater  than  2 (2  4 here) 
and  s is  a semi-saturation  constant  (0.0075). 
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The  contrast  detection  thresholds  used  in  the  legibility  model , shown  in  Figure  13,  are 
from  van  Nes  and  Bouman,  1967. 


Frequency  (c/d) 


Retinal 

Illuminance 

(td) 


Pupil  diameter  in  mm  were  attained  via  table  lookup.  The  values  used  were  for  viewing  a 
white  lambertian  surface  at  illuminances: 


3.426e-6  fed 
3.426e-4  fed 

3.426e4  fed 

s"^h^ 

used. 

The  depth  of  eye  in  mm,  from  exit  pupil  to  retina  was  set  at  20. 3,  an ^ 7 
The  cortical  scaling  parameter  was  set  accordingtojYap^viandKlein,  1987)  . . 

The  retinal  scaling  factor  was  set  according  to  (Watson,  1^84)  at  z.o. 

5. 2.2. 2. 2 Vision  Enhanced  Jack  Interface  Functions 

5.2.2.2.2.1  The  VVF  Display 

Thf  VVF  is  a three-dimensional  graphical  construct  In  the  WF  display,  three- 

«$££' *Sd  - . W US  each  has  a *«*  *“  COn,“S 

information  about  the  vertices,  faces,  and  location  of  the  object. 

As  the  WF  is  a three-dimensional  construct,  we  need  to  specify  projection  and  viewing 
Information  i vVF  is  displayed  as  a perspective  projection,  the  parameters  of  which  are 
55  S yTk'  view  reference; point  and  center  of  projecuon  are  also 

stored  in  the  window’s  database,  and  can  be  changed  tnteracuvely  to  look  at  any  point  in 
the  VVF  from  any  location. 
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Each  time  a new  WF  window  is  created,  the  window  is  assigned  default  values  for  the 
projection  and  view,  but  it  inherits  the  objects  that  already  exist  in  the  WF.  Essentially 
opening  a new  WF  window  gives  the  user  the  ability  to  view  the  WF  in  a different  way. 

S.2.2.2.2.2  The  Retins  Displ&y 

Theretma  display  illustrates  the  dynamic  relationships  of  three  types  of  objects  with  respect 

vvf  T>,Ud  aX1S;  By  dc/a,ult’  “ shows  retina1  images  that  are  formed  by  objects  in  A? 
VVF  The  orientation  of  these  images  varies  with  changes  to  the  fixation  point  and  with 
orientation  and  position  changes  of  the  head.  A subset  of  retinal  images,  that  form  the 
second  object  type,  are  formed  from  objects  that  are  fixed  with  respect  to  the  head  Since 
helmet  mounted  devices,  nose  bridge,  glasses,  etc,  are  stationary  to  the  head  coordinate 

°bj!Ct  typc1remiuns  stationary  in  the  retina  display  unless  the  fixation  point  is 
d?spl la?  can  dso  dlustrate  areas  that  are  characteristic  of  the  retina,  and 
^therefore  fixed  m location  and  orientation  even  when  fixation  and  the  head  position  is 
varied.  The  fovea  and  the  natural  blind  spot  are  examples  of  fixed  data.  (Wellrefer  to 
»w.dvn,aS  reQna  ?bject?  to  distinguish  them  from  the  variable  retinal  images  discussed 

S'pS's  EEflSSE?1 ,“0"sh,ps  bMween  “ 1 *» ' — 

The  user  can  open  additional  retina  windows  as  needed,  and  the  retina  displays  can  be 
interactively  customized  to  make  it  easier  to  see  data  of  interest.  For  example,  each  retina 
window  can  display  either  the  right  or  left  retina,  or  the  two  superimposed  on  each  other 

f dlsp.Iaycd  m Shades  of  green,  right  retina  data  is  displayed  in  shades  of 
red,  and  areas  of  overlap  are  shown  in  yellow. 

The  user  can  build  up  a database  of  retinal  objects  e.g.  cone  density  data,  iso-focus 
contour,  etc.  This  is  the  means  by  which  the  legibility  data  is  brought  into  VMT.  Retina 
objects  can  also  be  interactively  drawn  directly  onto  the  right,  left  or  both  retinas. 

Animponantdifference  between  retinal  images  and  retinal  objects,  is  that  the  objects  can  be 

5.2.2. 2.2. 3 Field  of  View  Cones 

can  **  “glc  for  the  field  of  view  of  interest  Semi-transparent  view 

cones  arc  then  projected  along  the  current  visual  axis  for  the  right  and  left  eyes  into  the 
crewstation.  The  convergence  of  the  cones  is  at  the  fixation  point.  The  intereection  of  these 
cones  and  the  crewstation  delineates  the  area  that  falls  within  the  current  field  of  view 

jCHiflESi 

5. 2. 2. 2.2.4  Total  Field  of  View  Plots 

Of  major  importance  in  vehicle  design  is  the  area  visible  out  the  window.  Total  field  of 

toC«£$ thC  dCSlgT  Wlth  a •3S,(?e*ree  PIot  of  cxtcmal  visibility.  It  is  possible 
to  read  information  eg.  over-the-nose  visibility,  directly  from  these  plots.  The  results  can 
be  seen  immediately  as  the  designer  explores  effects  of  various  body  types,  widows 
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5. 2.2. 2.2.5  Retrojections 

Retinal  objects  can  be  selectively  retrojected  into  the  WF.  By  retrojecting  a retinal  object, 
“ Se  wh^e  h intersits  the  VVF.  On  a computer  that  of 

mmsnarent  surfaces  the  retrojection  is  drawn  as  a semi-transparent  volume  in  the  VVF. 

Sf computers  that  don’t  support  transparentsurfaces,  °udU1Cd 

with  a series  of  rays  (lines)  that  are  drawn  from  the  eyeball(s)  out  into  the  VVK 

5.2.3  External  Interface  Detailed  Design 

Hie  vision  model  has  not  been  integrated  with  the  rest  of  simulation.^  the  next 

phase  it  will  follow  the  interfaced  standards  developed  for  MIDAS  communications. 

5.2.4  Coding  and  Implementation  Notes 

6.0  USER’S  GUIDE 

The  Vision  User  Guide  has  two  main  parts.  The  first  explains  how  to  use  the  David 
Samoff  Legibility  Model  ( LM ) software  to  produce  legibility 'data Tor  display  m Jack  The 
second  part  explains  the  use  of  the  vision  specify options iinfte  A Iww oi  Jack, 
hereafter  referred  to  as  the  Vision  Enhanced  Jack  Interface  (VKJ Revision 
enhancement  was  a result  of  a rewrite  of  an  earlier  program  called  VP  for  Volu™  . , 

Perimetry  from  the  Lighthouse  Research  Laboratory.  The  vision  options  are  not  standard 
SSfn^ly  released  with  new  vmionsof  Jack. They  «*>«». r^nnuned  or 
documented  by  the  University  of  Pennsylvania.  Further  releases  of  Jack  may  be 
incompatible  with  the  vision  options. 

6.1  OVERVIEW  OF  PURPOSE  AND  FUNCTION 

Geometrical  data  such  as  the  volume  field  of  view , occlusions,  facial  geometry  and  helmet 
margins  can  also  be  projected  into  the  cockpit  with  respect  to  the  coordinates  of  the 
aviator's  eyes  and  fixation  point . The  intersections  of  the  projections  with  objects  in  the 
crewstation,  delineate  the  area  of  coverage,  masking,  or  occlusion  associated  with  the 

objects. 

nhWK  in  the  crewstation  space  can  be  projected  onto  models  of  the  operator's  retinas. 
?£ to  provide  the  designer  with 

visual  ancles  subtended  by  objects  in  the  crewstation  space.  Both  the  nght  and  left  eye 
retinal  projection  s^re  mapped^.  The  retinal  map  is  yoked  to  the  fixation  point  and  changes 
asthefixatSn^Lus  interactively  manipulated.  Performance  contours  on  the retinas  can 
also  be  indicated  thus  aiding  the  designer  in  understand  the  limitations  to  visibility  imposed 
by  retinotopic  processing. 

6.2  INSTALLATION  AND  INITIALIZATION 

To  install  the  vision  enhanced  version  of  Jack,  follow  the  normal  installation  instructions 
orovided  with  Jack . There  are  some  additional  legibility  data  files  that  were  created  with 
the  Samoff  model  that  should  be  copied  to  the  same  directory  that  you  execute  Jack  frorm 
KS h^namS  simUar  to  Q<5jl  Jl.dl.cons  and  contain  the  precomputed  leg.bil.ty 

predictions  contours. 
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6.3  STARTUP  AND  TERMINATION 


For  startup  and  termination  instruction  for  the  vision  enhanced  version  of  Jack  see  the  Jack 
documentation.  The  LM  is  called  from  the  Unix  prompt  and  is  not  dependent  on  any 
environmental  variables.  To  start  LM  type  the  filename  ( cortex  ) followed  by  the  input 
files  ( pixel  descriptions  of  the  symbols  to  discriminate  between  followed  by  any  additional 
options.  It  will  run  to  completion  without  further  intervention. 


6.4  FUNCTIONS  AND  THEIR  OPERATION 
6.4.1  Sarnoff  Legibility  Model  Functions 

To  run  LM  execute  the  file  named  cortex.  This  should  be  in  the  directory  called 
$UPENN/4.5/legibility/src.  You  call  the  program  by  typing: 

cortex  < inputl  *1  lum  -i  ilium  -e  ecc  -d  dfix  -ds  dstim  input2 

Where: 

inputl  is  the  bitmap  image  file  for  the  first  image 
lum  is  display  luminance  in  foot-lamberts  (fL) 
ilium  is  ambient  illuminance  in  foot-candles  (fed) 
ecc  is  eccentricity  of  displayed  stimulus,  in  degrees 
dfix  is  the  fixation  distance  in  mm 
dstim  is  stimulus  distance  in  mm 

input2  is  the  bitmap  image  of  the  second  image  to  be  desciminated  from  the  second 
image.  If  there  isn't  a second  image  than  a second  copy  of  the  first  is  used. 

In  the  absence  of  an  explicit  input  parameter  the  following  defaults  are  used. 

illum=0.0,  ecc=0.0,  lum=10.0,  dfix  = 741.12,  dstim  = 741.12; 


a typical  call  would  look  like  this: 

cortex  < large  Q file  -1  10  -ilOO 

The  files  pyrg.fir  and  steer.fir  must  be  present  in  the  same  directory  as  cortex.  These  files 
are  used  as  date  for  the  filters  used  in  the  model. 

Here  is  what  the  output  of  cortex  means:  Without  the  -p  1 flag,  the  output  is  simply 
eccentricity  and  probability  correct  discrimination.  By  running  cortex  for  a range  of 
eccentricities  for  each  combination  of  luminance,  illuminance,  and  fixation  distance  It 
generates  the  kind  of  data  files  you  can  see  in  the  $UPENN/4.5/legibility/data/*.dat  files. 

When  you  run  cortex  with  the  -p  1 flag,  you  get  a lot  of  intermediate  results  that  can  be 
used  for  debugging.  On  the  first  line: 

veil  shows  the  screen  luminance  plus  the  output  of  veiling  illuminance  reflected 
off  the  screen. 

ret  shows  the  retinal  illuminance  in  trolands,  calculated  as  a function  of  pupil 
diameter  and  veil  (the  veiling  luminance), 
fcon  is  the  contrast  reduction  in  the  image,  resulting  from  the  veiling  luminance 
•scale  shows  the  inverse  of  the  retinal  and  cortical  magnification  factors 

needed  to  scale  the  frequency  and  size  of  the  sensors  in  the  periphery  of 
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the  visual  field. 

detector  output  space,  from  which  the  pmbabili  V 

is  calculated 

A 0, *eve* - - - — 
to  the  stimuli. 

The  files  containing  the  input  character^ So,  for  example, 
composed  of  5x5  Wb onvemion  I have"  used  in  the  model  m 

angle  corresponds  to  approximately  170  pixels. 

The  user  can  generate  contour  date  ( foi™tfo^  ^hVsame dh^oryas  cortex.’  You  Should 
piece  of  software  called  fcons.  Ttatpnm > » as 

££i;  3>  n is  ’the  number  of  team  dtc  input  data  file,  indat  ts 

the  input  data  file,  and  out.cons  is  the  output  file  of  contou  . 

You  call  it  as 

fcons  n < in.dat  > out.cons 

where  n is  the  number  of  lines  in  the  input  data  file,  iadat  is  die  input  data  file,  and 
out.cons  is  the  output  file  of  contours. 


Retina  Display 


i 

! 

• 

=> 

=> 

■bsebssbei 

• 

=> 


retina  display 


create  retina  window 

create  fiels  of  view  cones 
fixate  eyes  on  site 

interactive  fixation 

monocular/binocular  map 
zoom  in/out  of  retina  window 
show  eye  scan 


Figure  14.  Retina  Display  Menu 
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6.4.2  VEJI  Options 

6.4.2. 1 Retina  Display 

aSfiS?3SSSSlr^* 

o^isualS  of  coverage 

6.4. 2.1.1  Create  Retina  Window 

To  view  the  environment  as  seen  from  the  aviator’s  eves  selwt  th#.  w0.- • .,  » 

sa:^  s wofa*  ofT"8  °f  ■ ^ in 

FSS~!F^ 

eye  g }'  hC  red/green  d,sk  rePresents  the  biological  blind  s,St  for  the  left  or  right 

6.4.2.1.2  Create  Field  of  View  Cones 

agKSi  <S  2L^SK5Kv2SSsSi"  'wo  ^r*'5 

SSSSS-SSSS^*^ 

than  one  type  of  view  cone  if  desired.  Y 1 “ deleted  You  "^y  project  more 

6.4.2. 1.3  Create  Monocular  Field  of  View  Cones. 

SSSS5SlS-'l™™^ 

project  an  arbitrary  angled  field  of  view  cone.  The  angle  is  sefbyuser  in  tte  vfsion  *** t0 

srasss  f i-s  s sS"?^;s  option- 
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6.4. 2.1.4  Fixate  eyes  on  site 

This  ^^^^SneifSeMcrcMa^muid^* 

6.4.2. 1.5  Interactive  fixation 

2&3EsS3&£&£!^s? 

JSS  rroS  obTc:  in  Jack* See  the  Jack  User  Manual  for  more  details. 

6 4.2.1.6  Monocular/Binocular  Map 

tag!:  S°K1  the  option  again  returns  the  users  to  projecttng  both  eye  tages. 

6.4.2. 1.7  Zoom  in/out  of  retina  window 

Selectine  this  option  zooms  the  retina  Display  in  so  that  only  the  center  30  degrees  are 
vSKKe  is  heipful  for  seeing  deuul  about  objea  sis. = artreta  locatton. 
Selecting  this  option  again  returns  the  user  to  display  the  full  90  degre  . 

6.4.2. 1.8  Show  eye  scan 

™heM°^ 

SSg'obj^W  this  ffuncS  a futurc  relCaSC' 
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Retina  Editor 


Sea 

option  menu 

=> 

• 

=> 


1 retina  editor  | 

create  retina  editor 

draw  left  retina  object 

draw  right  retina  object 

draw  filled  retina  objects 

retroject  left  retina 

retroject  right  retina 

retroject  both  retina 

clear  both  retina 

dear  retina  objects 

save  retinda  objects 

load  retina  objects 

load  QO  confusion  data 

load  qo  confusion  data 

load  Iso  Focus  contour  data 

load  cone  density  data 

zoom  In/Out  of  editor  window 

Figure  15.  Retina  Editor  Menu 

6.4.2.2  Retina  Editor 

See  Figure  1 5 for  an  overview  of  the  functions  available  under  this  option. 
6.4.2.2.1  Create  Retina  Editor  Window 


Tlie  retina  editor  window  provides  a way  to  plot  visual  relevant  performance  data  in  a fhvM 
centric  polar  coordinates  map,  similar  to  the  retina  display  Whatever  i«  drawn  nr  ■ 

A, s window  can  also  be  projected  into  the  enviXnS 

se  ect  this  option.  The  normal  SGI  window  opening  and  position  is  then  performed  The 
user  should  try  to  posiuon  this  window  some  place  where  it  does  not  occlude  useful 

the  user  can  always  resize  the  Jack  screen  SSSg  sS  titat  is 
appropriate.  The  editor  window  is  not  an  integrated  Jack  window  and  only  gets  updated 
when  an  event  occurs  that  would  change  what  is  displayed  in  the  window* 8 ^ 

6.4. 2.2. 2 Draw  left/right  retina  object 

0pt1.01? 1S  used  t0  f^aw  arbitrary  contours  onto  the  retina  editor.  After  selecting 
tw**  Cft  ?r.nght  draw  °P»ons  move  the  mouse  cursor  over  the  point  on  the  editor  window 
that  you  wish  to  enter  as  the  first  point,  press  the  left  mouse  button.  Now  move  to  the  next 
point  and  press  the  left  mouse  button.  Continue  entering  points  until  you  have  entered  all 
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the  data  points  and  then  press  the  right  mouse  button  to  close  the  contour.  The  contour  will 
be  colored  red  for  right,  green  for  left  retina  object . 

6.4. 2.2. 3 Draw  filled  retina  objects 

This  notion  toggles  back  and  forth  between  drawing  die  data  as  Unes  or  polygons.  The 
polygons  are  semi  transparent  and  interior  contours  can  be  distinguished  from  overlapping 

polygons. 

6.4.2.2.4  Retroject  left/right/both  retina 

This  ootion  turns  on  the  retrojection  ( projection  into  Jack's  environment)  of  data  loaded  or 
drawn  onto  the  retina  editor.  The  projection  of  this  date  is  along  the  visual  axis  of  the  figure 
rdTxtendsm  the  plane  positioned  at  the  fixation  point  and  normal  to  the  visual  axis. 

6.4.2.2.5  Clear  retina  objects 

Selection  of  this  option  clears  all  data  on  the  retina  editor. 

6.4. 2.2.6  Save/Load  retina  objects 

Selection  of  these  options  allows  a user  to  save  the  data  that  has  been  drawn  onto  the  editor 
or  load  in  new  data. 

6. 4. 2. 2. 7 Load  QO/qo  confusion  data 

These  options  are  a special  case  of  the  above  load  option.  They  were  put  on  the  menu  only 
tonSceheasier  to  demonstrate  the  vision  model.  Instead  of  taking  as  input  a filename 
this  option  builds  the  correct  file  name  for  the  current  conditions  The  conditions  thateffect 
the  file  that  is  load  are:  ambient  illumination,  stimulus  lumens  and  character  size.  Light  are 
set  in  the  vision  state  information  settings.  Character  size  is  detemuned  by  your  selection  of 
large  ( QO ) or  the  small  (qo)  letters  in  the  option  name.  If  any  of  the  light  settings  are 
changeStenhe  data  is  lolled  you  must  clear  the  editor  of  the  old  data  and  reload  the 

confusion  data  again. 

6.4.2.2.8  Load  Iso  Focus  / Cone  density  contour  data 

This  option  is  the  same  as  the  above  options  for  the  loading  of  confusion  data,  exceptthat 
the  data  does  not  depend  on  the  ambient  illumination  and  stimulus  lumens  and  therefore 
doesn't  need  to  be  cleared  and  reloaded  when  they  change. 

6.4.2.2.9  Zoom  in/out  of  editor  window 

This  option  zooms  in/out  of  the  retina  editor  window.  Some  of  the  perfonmn<re  data  can 
cover  only  a small  portion  of  the  retina  window  when  this  occurs  it  is  useful  to  zoom  in. 
Selecting  this  option  again  zooms  out. 
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Vision  Information 


main  menu 


option  menu  => 


option  menu 

vision  information 

• 

create  adaption  info  window 

vision  information 

create  legend  window 

create  Aitoff  window 

L • 1 

initialize  state  info 

set  state  information 

Figure  16.  Vision  Information  Menu 


6.4.2.3  Vision  Information 

See  Figure  16  for  an  overview  of  the  functions  available  under  this  option. 

6.4. 2.3.1  Create  adaptation  info  window 

This  option  reflects  work  in  progress.  The  purpose  for  this  window  is  to  show  the  current 
state  of  the  observer.  As  factors  such  as  ambient  lighting  changes  the  observer  state  is 
assumed  to  be  adapted  instantaneously.  The  assumed  current  state  is  sometimes  of  interest 
for  the  user.  It  may  also  be  useful  to  investigate  mis-adaptive  states. 

6.4.2.3.2  Create  legend  window 

This  window  will  be  used  to  provide  a legend  for  color  used  in  the  display  of  data. 
Generally  speaking  green  is  associated  with  attributes  of  the  left  eye,  red  with  the  right  eye 
and  yellow  with  both.  The  idea  is  to  provide  a standard  way  to  communicate  the  meaning 
of  complex  data  in  a consistent  way.  This  window  is  not  completed  at  this  point 

6. 4. 2.3.3  Create  Aitoff  window 

Aitoff  charts  arc  used  widely  by  industry  to  standardize  visual  parameters  based  on  the 
coordinates  of  the  design  eye.  To  use  this  option  the  user  must  now  the  cooidinate  of  the 
design  eye  and  define  a site  at  that  location.  The  user  can  then  get  an  Aitoff  graph  relative 
location.  This  options  plots  everything  in  the  environment  and  therefore  produce  a very 
cluttered  plot. 

6.4.2.3.4  Initialize  state  information 

This  option  should  always  be  done  before  attempting  to  use  any  of  the  vision  options.  It  is 
best  to  include  this  in  the  startup  script  for  the  Jack  environment. 

6.5  ERROR  AND  WARNING  MESSAGES 

Error  and  Warning  messages  are  done  in  the  same  way  Jack  handles  them.  See  Jack  for 
more  information. 
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6.6  RECOVERY  STEPS 

Recovery  Step  are  done  in  the  same  way  Jack  handles  diem.  See  Jack  for  more 
information. 

7.0  ABBREVIATIONS  AND  ACRONYMS 

MFD  Multi-Function  Display 

VVF  Volume  Visual  Field 

MIDAS  Machine  Integration  Design  Analysis  System 

JACK  Not  an  acronym  ( just  looks  like  one ) 

VEJI  Vision  Enhanced  Jack  Interface 

VP  Volume  Perimetry 

LM  Legibility  Model 


8.0  GLOSSARY 

9.0  NOTES 

9.1  FUTURE  DIRECTIONS 

9.1.1  Illumination  Model 

Illumination  modeling  is  undergoing  rapid  progress  towardsmorei^^ 
models  There  are  many  tradeoffs  in  the  illumination  algorithms  between  accura  y, 
c^Dutationtinw  and  noticeable  differences.  Emphasis  can  be  placed  on  reaching  pleasing 
Sstk resS™  thus  providing  an  illusion  of  reality.  The  algorithms  implicitly  consider 
SmSTthrdisplay  median  to  limit  the  amount  of  computation  to  diat  which 
differences  are  detectable  on  most  graphic  displays.  Emphasis  could  be  shifted  to  mode 
the  laws  of  physics  that  rigidly  govern  the  propagation 

accurately  as  possible.  Our  vision  model  will  require  a high  degree  of  fidelity  to  tne 
physical  worldto  limit  the  eirors  in  other  dependent  computations^  Many  of  the  techniques 
to  follow  will  not  execute  in  anywhere  near  real-time.  They  will  however,  have 
intermediate  results  that  may  be  used  for  displaying  approximate  graphics. 

The  accurate  presentation  of  data  is  confounded  by  the  limited  display  capabilities  of  die 
CRT  A CRT  is  capable  of  displaying  only  a subset  of  colors  and  intensities  rai}ges. 
also  of  finite  resolution.  What  can  be  presented  on  a CRT  is  not  the  same  stimuh  that  would 
be  experienced  if  one  were  actually  there.  For  this  reason  the  graphic  image  willonly  be 
used  for  rough  approximations  where  a large  number  of  possible  “lutl°9s  ^er^f.S 
evaluated.  After  selection  of  a specific  solution  it  can  be  reevaluated  at  higher  fidelity. 

The  development  of  an  accurate  illumination  model  has  a profound  impact  on  the  rest  of  the 
simulation.PThe  physical  response  to  electromagnetic  waves  of  die  surface  matenal  of 
every  object  in  the  simulation  must  be  known.  The  orientation  of  all  objects 
to  calculate  the  visibility  to  illumination  sources  and  the  viewer.  Anenuau°n  fact^  of  the 
environment  in  which  the  viewer  and  illumination  sources  are 

included  in  the  model.  Attributes  of  the  illumination  sources  must  also  be  ir^eleri.  Tliis 
implies  that  every  object  ( even  geometrically  simple  ones ) will  need  a relatively  large 
amount  of  illumination  data  in  addition  to  the  normal  geometric  data. 

The  theoretical  basis  for  the  illumination  model  will  be  a hybrid  of  several  c^plementary 
techniques.  Forming  the  basis  for  the  modeling  of  diffuse  illumination  wdl  be  that  of  a 
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hemispherical  radiosity  model.  This  model  was  first  used  to  study  heat  transfer  between 

tttXZSfr. OT  °v  8 Spakcccraft-  ?c  “d*0***  has  been  recently  extended  by 
Goral  at  Cornell  University  to  the  area  of  computer  graphics.  Other  researchers  Cohen  * 

^ ^ **-  *0  handl^ial^s^ SSty 

gradient  artifacts  found  in  Goral  s original  model.  y 

The  radiosity  model  provides  a solution  for  diffuse  and  emissive  radiant  energy  but  does 
not  model  specular  reflectance  and  refraction.  For  this  type  of  illumination  aray  tracing 
SEuh1011  appropriate.  This  model  simulates  lights  with  rays  and  propagates  them 
throughout  the  environment.  Surfaces  reflect  or  transmit  these  rays  depending  on  their 

fc°rPH  ?tfnbHtes-  Pl?Pcr  attention  is  paid  to  angle  of  incidence  and  refraction!  Ray  tracing 
is  CPU  intensive  and  therefore  will  be  used  for  final  analysis  rather  than  for  interactive  8 
evaluation  of  the  design  solution  space. 

Roy  Hall's  illumination  model  presented  in  his  took,  Illumination  and  Color  in  Computer 
Generated  Imagery,  is  the  most  complete  illumination  model  to  date.  Glassner  ( An 
introduction  to  Ray  Tracing)  codifies  Hall's  model  and  suggests  how  to  extend  it  to 

tran.s™sslon  of  u8ht:  < appendix ) This  is  the  illumination  model  that 
i ^?ht  parameters  in  our  vision  model.  The  error  between  Hall's  model 

ri?flti.nri1CMi1HGfaph-C  S ught  modcl.(  ^ appendix ) wiU  be  explored.  The  size  of  this 
solutions6  Wl  determine  Ae  appropnateness  of  this  model  for  use  in  approximating 

Special  procedures  can  be  provided  to  model  local  phenomena  that  can  be  precomputed 
For  example,  the  glare  due  to  the  sun  could  be  precomputed  for  various  angles  and  a set  of 
light  masks  could  be  overlaid  on  a CRT  to  model  the  expected  glare. 

to  computed8  PrOVidC  thC  data  structures  ncce«ary  for  the  afore-mentioned  algorithms  to 

General  Illuminators 

This  module  contains  the  data  structures  and  access  functions  for  the  illuminator  data 
nwfni^n  T3  TC  mV  1 could  3150  M by  Ac  output  of  some  other  simulation 

W « as ffolfeT"”0"  supplied  bom,lKtES  dng  Handbook. 

Direction  Vector 
Energy  Spectrum 
Characteristics 
Location 
Geometry 
Reflector  type 

Sun,  Moon 

The  sun  and  moon  are  special  case  of  illuminators  and  data  will  be  attained  from  a TBD 
source. 

Flares,  Lasers 

Flares  and  lasers  present  some  special  problems  in  vision  and  need  to  be  addressed 
differently  in  the  model.  These  data  will  to  very  time  dependent  in  both  the  spatial  and 
amplitude  domains  The  effects  of  narrow  frequency  ranges,  strong  lighting  contrast  and 
filters  make  these  illumination  sources  difficult  to  model.  ® 
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Light  Loss  Factors  ...  . . f 

This  module  contains  the  data  structure  and  functions  to  model  loss  of  light  from  someot 
the  major  causes.  Shadows  will  comprise  a significant  portion  of  the  light  loss  cases.  Tnese 
cases  are  addressed  as  part  of  Hall's  illumination  model.  The  others  will  be  modeled  from 
data  out  of  the  IES  Lighting  Handbook. 


Shadows  types 

The  major  types  are  shadows  are:  Umbras,  Penumbras,  Geometric 


Attenuation 


Non  Recoverable  Factors  . ...... 

The  non  recoverable  factors  are  those  features  of  luminaries  that  cause  deviations  from  the 
controlled  laboratory  but  are  not  corrected  by  light  maintenance  procedures.  The  IES 
Lighting  Handbook  identifies  these  factors  as  important  when  computing  light  calculations 
and  states  that  they  are  multiplicative.  The  total  light  loss  is  the  product  of  these  factors. 
Temperature  Factor,  Line  Voltage  Factor,  and  Surface  Depreciation. 


Recoverable  Factors  . . , 

The  recoverable  factors  are  like  non-recoverable  factors  with  the  exception  that  they  can  be 
corrected  by  proper  light  maintenance.  Some  examples  are:  Lumen  Depreciation  Factor, 
Dirt  Depreciation  Factor,  Lamp  bum  out  Factor. 


Workspace  Model  ....  ..  . . . . .. 

The  Workspace  model  is  comprised  of  the  data  description  for  the  objects  that  exist  in  the 
environment  In  addition  data  about  the  current  state  of  the  environment  is  maintained. 
Objects  are  a generic  name  for  an  entity  in  the  simulation.  An  object  can  be  used  as  a stimuli 
to  the  vision  model.  An  object  has  at  least  the  following  slots: 

The  orientation  and  positional  data  are  maintained  in  world  coordinates  for  each 
object. 

A material  definition  is  provide  for  each  polygon  that  comprises  an  object.  This 
material  definition  is  needed  for  the  radiosity  model.  Material  properties  include. 
Emission  color.  Ambient  reflectance.  Diffuse  reflectance.  Specular  reflectance, 
Shininess  ( specular  light  scattering  exponent ) Alpha  ( transparence)  and  color. 

A geometric  description  including  information  such  as  vertices  coordinates  is 
maintained  for  all  objects. 

Many  objects  in  the  environment  can  exist  in  various  states.  For  those  objects  that 
posses  this  characteristic  the  associated  state  variables  are  maintained  in  the  object 
data  structure. 

An  important  factor  for  target  detection  is  Atmospheric  Attenuation.  The  visibility  of  a 
distance  object  is  very  much  dependent  on  conditions  in  the  atmosphere.  The  predominant 
influencing  factors  are:  Precipitation,  fog,  clouds,  smoke,  haze,  etc.  Attributes  and 
procedural  descriptions  will  be  stored  here  to  describe  these  effects. 

Texture  Maps  are  required  for  added  accuracy  for  many  of  the  detection  algorithms.  An 
example  of  where  texture  maps  may  be  used  would  be  target  masking.  Many  targets  use 
some  form  of  camouflage  to  make  detection  more  difficult.  It  will  be  important  to  model 
target  masking  to  attain  accurate  results.  The  problem  of  detection  is  more  general  than 
intentional  masking.  The  detectability  of  any  object  depends  to  a large  extent  on  the  contrast 
between  itself  and  its  background.  Texture  maps  are  necessary  to  generate  realistic 
backgrounds,  eg.  terrain  or  sky,  from  which  an  object's  detectability  will  calculated. 
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— SARNOFF  LEGIBILITY  MODEL 
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Sarnoff  Cockpit  Display  Visbility 
Modelling  for  NASA  A3I  Project 

Jeffrey  Lubin  James  R.  Bergen 
November  13,  1990 


1 The  Task 

SamofTs-  task  in  the  A3I  project  is  to  develop  a quantitative,  easily  com- 
putable model  of  instrument  visibility  within  an  aircraft  cockpit  environment. 
The  purpose  of  the  model  is  to  provide  visibility  data  ^ display  designers 
who  need  quantitative  information  on  the  effect  that  their  desi^1C«^ 
have  on  crew  performance  over  a broad  range  of  mission  scenarios.  Given 
this  range,  the  visibility  of  alphanumeric  and  other  information  depends i n 
orlv  on  the  spatial  configuration  of  the  display,  but  ako  importantly  on  light- 
tag  parameters  such  as  light  level  in  the  cockpit  and  the  observer  state  of 
adaptation.  The  model  must  therefore  accurately  assess  the  =ff«t  of  such 
parameters,  and  convey  this  information  to  the  drsplay  desrgner  m sn  easily 
understandable  form. 


2 Background 

A number  of  visibility  models  have  already  been  developed  by 
both  the  applied  vision  and  basic  psychophysics  communities, 
successfully  predict  human  performance  within  a restricted  range 
and  task  domains. 


workers  in 
Each  can 
of  stimulus 
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2.1  Applied  psychophysics 

An  early  success  in  applied  vision  was  the  JND  Model  of  Carlson  and  his  as- 
sociates (Carlson  and  Cohen,  1974;  Carlson  and  Klopfenstein,  1985).  In  this 
model,  an  input  image  is  decomposed  via  a one  dimension  *1  fourier  trans- 
form into  a number  of  spatial  frequency  bands.  These  filtered  bands  are 
then  perturbed  by  various  noise  sources,  squared,  and  spatially  integrated. 
Changes  in  the  output  of  this  process  from  one  member  of  a pair  of  images  to 
the  other  provide  a simple  perceptual  measure  of  the  visibility  of  differences 
between  the  two  images.  This  model  has  successfully  predicted  the  visibility 
of  changes  in  edge  sharpness  and  of  various  display  artifacts,  among  other 
things.  The  disadvantages  of  the  model  are  that  it  is  spatially  one  dimen- 
sional, and  is  somewhat  complicated  to  compute,  since  a noise  parameter 
must  be  adjusted  for  each  change  in  display  parameters  such  as  luminance 
and  display  size.  A more  recent  variant  of  this  model,  the  Barten  (1987) 
SQRI  Model,  solves  some  of  the  complexity  problems  by  introducing  poly- 
nomial approximations  for  the  changes  in  human  sensitivity  to  changes  in 
display  parameters. 

2.2  Basic  psychophysics 

Similar  models  have  been  introduced  into  the  basic  psychophysics  literature 
by  Wilson  and  his  colleagues  (e.g.,  Wilson,  McFarlane,  and  Phillips,  1983; 
Wilson  and  Regan,  1984),  based  on  the  threshold  model  of  Wilson  and  Bergen 
(1979),  and  by  Legge  and  Foley  (Legge  and  Foley,  1980;  Foley  and  Legge, 
19S1).  These  models  successfully  predict  human  performance  in  simple  psy- 
chophysical tasks  such  as  grating  contrast  detection  and  discrimination.  In 
all  of  these  models,  the  input  image  is  first  decomposed  into  independent 
spatial  frequency  channels  by  a set  of  linear  filters.  The  output  of  each  filter 
is  then  put  through  a sigmoid  non-linearity,  the  shape  of  which  matches  very 
closely  that  of  the  non-linearity  imposed  by  the  noise  and  squaring  steps  of 
the  JND  Model. 

2.3  Two-dimensional  models 

All  the  models  described  above  are  spatially  one- dimensional;  that  is,  they 
predict  sensitivity  to  spatial  variation  in  one  dimension  only.  Watson  and 
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his  coUepgp*.  (Watson,  1983;  Ahumad.  and  Watson,  1988;  Njdam.  Wat- 
son,  and  Ahnmada,  1985)  have  imp  emented  a modd  wh.eh  jraeralines 
Unear  filtering  stage  of  these  models  to  two  dimensions.  Each  filter  is  a 
two-dimensional  gabor  function,  with  a number  of  different  scales,  orienta- 
tions, and  phases  of  filtering  at  each  point  in  the  two-^^ion^wsl  field, 
and  an  increase  in  the  overall  scale  of  filtenng  as  a function  of  ec^tnaty. 
The  model  has  been  validated  on  some  detection  and  discrimination  data. 
One  limitation  is  that  it  is  only  accurate  at  stimulus  ^ ^ar  detecrion 
threshold  since,  unlike  the  other  models  described  above,  there  is  no  point 
non-linearity  after  the  Unear  filtering  stage. 

2.4  Combining  information  across  channels 

One  problem  all  these  models  must  face  is  how  to  combine  information  across 
a large  number  of  different  filtering  channels,  so  that  a uni-dimensional  val 
for  human  performance  on  a discrimination  task  can  be  obtained  In  other 
words  from  the  large  dimensional  space  represented  by  the  channel  outpu  , 
the  models  must  derive  something  like  a single  value  for  the  probability  of 

detecting  a difference  between  a pair  of  stimuli. 

One  way  to  perform  this  reduction  of  dimensionality  is  to  base  the  per- 

formance  ™ only  op  the  single  chapno.  which 

change  in  output  from  one  member  of  the  stimulus  pair  to  the  other.  This 
"maximum-of”  decision  rule  is  implicit  in  most  of  the  basic  psychophysics 
modelling,  and  is  motivated  by  the  hope  that  the  simple  stimuli  usually ' used 
in  that  paradigm  are  sufficiently  localized  in  the  space  of  channel  out^ts  * 
that  only  one  channel  governs  performance,  regardless  of  the  degree  to  whi 

channel  outputs  axe  combined.*  . 

Other  models,  like  the  Watson  model,  use  variants  of  an  optimal  Bayesian 
classifier  at  the  channel  combination  stage.  Given  the  assumption  that  each 
channel  output  is  perturbed  by  zero  mean,  unit  variance  Gaussian  noise 
the  detectabiUty  of  a pattern  by  an  optimal  observer  is  directly  ProP°^°n 
to  the  euclidean  distance  of  the  pattern’s  feature  vector  from  the origin  of 
the  channel  output  space.  For  an  uncertain  observer,  detectability  can  be 
modeled  in  the  same  way,  but  with  an  exponent  higher  than  the  euclidean  2. 
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3 SarnofF  Approach 

3.1  Basic  strategy 

Scoff’s  approach  to  the  NASA  legibility  modeling  task  has  been  to  augment 
the  one-dimensional  discnmination  models  of  Carlson,  Barten,  and  the  basic 
psychophysics  community  to  include  the  two-dimensional  spatial  processing 
used  in  the  Watson  detection  model.  In  addition,  because  the  NASA  task 
requires  performance  prediction  over  a wide  range  of  lighting  conditions  and 
server  perceptual  states,  the  Sarnoff  model  includes  a front  end  that  pre- 
processes  the  input  images  to  model  the  effects  of  changes  in  illuminance 
screen  luminance,  and  observer  fixation  location  in  three-dimensional  space.’ 

3.2  Choice  of  performance  measure 

The  performance  measure  chosen  for  the  Sarnoff  model  is  probability  of  a 

numl  d,ShCnm;natl°"  J>etween  two  input  images;  e.g.,  between  two  ilpha- 
umenc  characters.  Other  performance  measures  are  of  course  possible^one 
common  measure  from  the  applied  psychophysics  community  is  image  Qual- 
ity, expressed  m units  of  just-noticeable  differences  (JNDs)  between  the  two 
images.  In  fact,  the  probability  measure  used  in  the  Sarnoff  model  is  derived 
from  a JND-like  measure  as  will  be  described  below.  Another  potentially 
useful  measure  is  the  reaction  time  required  for  correct  discrimination  How- 
ever  this  measure  has  received  little  attention  to  date  in  psychophysics  re- 

Xtion  rdXwould  thus  require  a Iarge  “ of  data 

• imP°rtant  consideration  in  the  formulation  of  the  Sarnoff  model 

is  that  the  performance  measure  be  conservative.  That  is,  display  design 
engineers  need  a measure  that  would  allow  them  to  rule  out  very  bad  designs 
but  would  let  marginal  designs  pass  for  additional  testing.  In  other  wofds’ 

th^  errf°n  thC  Side  °f  overestimating  ra^her  than  underestimating 

the  probability  of  successful  discrimination.  g 

3.3  Incremental  improvement 

Finally,  the  model  has  been  constructed  in  such  a way  as  to  allow  incremen- 
tal improvement  in  its  ability  to  predict  performance  among  complex  stimuli. 
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For  example,  the  model  as  currently  formulated  can  accurately  predict] perto- 
mance  only  among  static,  monochromatic  images.  However,  y rep  g 
model’s  spatial  filters  with  a set  of  spatio-temporal  filters,  performance  mea- 
surement among  moving  stimuli  would  be  possible.  The  strategy  in  model 
development  is  therefore  to  validate  the  simple  model  on  simple  stimulated 
then  incrementally  build  in  model  complexity  to  handle  the  more  complex 

stimuli. 


4 The  Model 

Figure  1 is  a schematic  diagram  showing  the  different  stages  of  the  visibility 
model.  In  this  section,  each  stage  of  the  model  will  be  described  in  detail. 

4.1  Input  parameters  and  stimulus  format 

The  model,  as  currently  formulated,  takes  as  input  one  or  two  image  Wes, 
and  a number  of  optional  lighting  and  observer  state  parameters  These 
parameters  are  listed  here  with  the  default  value  and  units  in  parentheses. 

Screen  luminance  (10.0  foot-lamberts) 

Illuminance  (0.0  foot-candles) 

Eccentricity  of  displayed  stimulus  (0  degrees) 

Fixation  depth  (741.12  mm) 

Stimulus  depth  (741.12  mm) 

The  image  files  slait  with  two  four-byte  integers  indicating  the  width 
and  height  of  the  image  in  pixels,  followed  by  rows  of  pixel  tmtoei .ml Soa£ 
The  images  can  be  any  size,  although  should  be  at  least  256x“6  toall 
filtering  within  a large  enough  range  of  different  frequency  bands.  The  con- 
version factor  from  pixels  to  mm  is  currently  fixed  in  the  software  as  13.2 
pix/mm.  With  only  one  image  as  input,  the  software  calculates  the  probabih 
fty  of  detecting  that  stimulus;  with  two  images,  the  standard  drscnmination 

probability  is  calculated.  . , . 

In  the  current  version  of  the  model,  pixel  values  are  required  to  range 

from  -1.0  to  1.0,  with  the  maximum  absolute  value  indicating  the  contrast  of 
fhe  stimulus.  For  example,  a sine  grating  stimulus  with  peaks  at  ±0^  would 
have  a contrast  of  0.5.  To  remove  the  need  for  this  convention,  a model  stage 
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Figure  1:  Visibility  model  flow  diagram. 
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which  computes  contrast  from  arbitrarily  scaled  input  images  is  needed,  but 
has  not  yet  been  implemented. 

4.2  Front  end  calculations 

Several  initial  transformations  on  the  input  images  axe  performed  prior  to  the 
linear  filtering  stage  of  the  model.  These  transformations  model  the  effect  of 
changes  in  fixation  depth,  veiling  luminance,  and  fixation  eccentricity. 

4.2.1  Fixation  depth 

In  order  to  account  for  changes  in  effective  image  resolution  with  changes  in 
the  difference  between  image  depth  and  fixation  depth,  we  used  geometrical 
optics  to  calculate  the  size  of  the  blur  circle,  and  then  pre-filtered  each  input 
image  with  this  disk-shaped  convolution  kernel.  This  calculation  requires 
knowledge  of  the  distance  from  the  exit  pupil  to  the  imaging  surface  (i.e., 
the  retina),  which  we  took  as  20.3  mm  from  Westheimer  (1986)-  It  also 
requires  an  estimate  of  pupil  size.  For  this,  we  wrote  a simple  interpolation 
routine  to  estimate  pupil  diameter  at  any  light  level  from  a table  pubhshed 
in  Hood  and  Finkelstein  (1986). 

4.2.2  Contrast  reduction 

Veiling  luminance,  caused  by  the  reflection  of  ambient  light  by  the  display 
screen,  reduces  the  effective  contrast  of  displayed  information.  We  are  mod- 
elling the  screen  face  as  a perfectly  lambertian  surface  with  a reflectivity  of 
10%.  This  assumption  implies  that  an  illuminance  of  10  fed  will  result  in  a 
veiling  luminance  of  1 fL.  We  are  defining  contrast  as 

c = (lmax  — lmin)/(lmax  + lmin) 

where  lmax  and  lmin  are  the  maxmimum  and  minimum  displayed  lumi- 
nances. Given  this  definition,  the  addition  of  a veiling  luminance  v to  both 
lmax  and  lmin  changes  the  contrast  by  a factor  l/2u. 

4.2.3  Eccentricity  scaling 

Abundant  psychophysical  evidence  shows  that  contrast  sensitivity  remains 
roughly  constant  across  the  visual  field,  if  the  grating  patch  is  scaled  up  by 
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a linear  function  of  eccentricity.  This  strongly  suggests  that  processing  is 
similar  across  the  visual  field,  except  for  a linear  scaling  up  of  weuaor  size 
towards  the  periphery.  In  order  to  model  the  effect  of  this  change  in  sensor 
size,  we  found  it  more  convenient  to  scale  down  the  size  of  the  input  images 
as  a function  of  eccentricity,  rather  than  to  scale  up  the  size  of  the  sensors. 
We  used  the  scale  factor  (*)  of  0.4  quoted  by  Watson  (1983),  so  that  the 
scaling  of  input  images  as  a function  of  eccentricity  (e)  is 

1.0/(1.0  + ke). 


4.3  Linear  filtering 

4.3.1  Pyramid  decomposition 

In  order  to  filter  within  a range  of  different  frequency  channels,  the  input 
image  is  first  decomposed  with  a gaussian  pyramid  into  channel*  spearated 
from  each  other  by  one  octave,  The  frequencies  we  chose  for  these  chan- 
nels are  identical  to  those  used  by  Watson  (1983);  i.e.,  32  through  0.5  c/d, 
corresponding  to  seven  octaves  or  equivalently,  seven  pyramid  levels. 

4.3.2  Computing  filter  gains 

The  human  visual  system  is  not  equally  sensitive  at  all  frequencies.  A plot  of 
contrast  detection  threshold  as  a function  of  spatial  frequency  shows  roughly 
an  inverted-U  shape,  with  a peak  at  roughly  2 c/d,  and  complete  loss  of 
sensitivity  by  approximately  60  c/d.  Moreover,  as  shown  by  van  Nes  and 
Bouman  (1967),  the  shape  of  this  contrast  sensitivity  function  changes  with 
retinal  illuminance.  To  model  these  dependencies,  the  image  component  in 
each  frequency  channel  is  weighted  by  a gain  factor  appropriate  for  the  retinal 
illuminance,  before  any  oriented  filtering  is  performed. 

Retinal  illuminance  (in  photopic  trolands)  is  calculated  as  the  amount  of 
light  incident  on  the  cornea  (in  cd/m*)  times  the  pupil  area  (in  mm*),  where 
the  light  incident  on  the  cornea  is  assumed  to  be  the  screen  luminance  plus 
the  veiling  luminance,  appropriately  converted  from  fL  to  cd/m*.  The  gain 
at  each  frequency  is  then  calculated  directly  from  the  van  Nes  and  Bouman 
data  with  a simple  log  interpolation  function  to  return  sensitivities  at  retinal 
illuminances  other  than  those  reported  in  the  data.  For  example,  if  the 
threshold  modulation  for  a 1 c/d  grating  were  1%  at  10  tds  and  5%  at  1 
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tds,  lien  w.  interpolate  the  threshold  at  3.16  Ids  (half  the  log  distance  &om 
1 to  10)  to  be  2.23%  (half  the  log  distance  from  1%  to  5%).  This  direct 
calculation  is  possible  only  under  the  assumption  of  no  summation  among 
different  frequency  channels;  any  assumed  summation  would  require  a more 
complicated  relationship  between  the  contrast  sensitivity  function  and  the 
gain  of  each  channel. 


4.3.3  Steerable  filtering 

For  convenience  and  speed  of  oriented  filter  operation,  we  use  the  steerable 
filters  of  Freeman  and  Adelson  (1990),  which  allow  separable  calculation  of 
linear  filter  responses  at  any  orientation  and  phase.  The  filters  implemented 
here,  a second  derivative  of  a gaussian  and  its  Hilbert  transform,  have  a log 
bandwidth  at  half  height  of  approximately  0.7  octaves.  This  is  within  the 
range  of  bandwidths  inferred  psychophysically  (e.g.,  Watson  and  Robson, 
1981).  Wp  are  using  four  orientations  (0,  45,  90,  and  135  degrees)  and  two 
phases  (sine  and  cosine),  four  a total  of  eight  oriented  filter  responses  per 
pyramid  level.  The  orientation  bandwidth  of  these  filters  (i.e.,  the  range  of 
angles  over  which  the  filter  output  is  greater  than  one  half  the  maximum)  is 
approximately  65  degrees.  This  figure  is  slightly  larger  than  the  40  degree 
tuning  of  monkey  simple  cells  reported  by  Devalois  et  al  (1982  , and  the  30 
to  60  degree  range  reported  psychophysically  by  Phillips  and  Wilson  (1984). 


4.3.4  Energy  calculation 

During  the  early  stages  of  model  testing,  we  found  that  the  detectability  of  a 
simple  edge  could  change  dramatically  with  small  changes  in  edge  position. 
To  combat  this  problem,  a small  amount  of  spatial  summation  was  added  by 
computing  energy  after  the  linear  filtering  stage.  That  is,  corresponding  sine 
and  cosine  filter  responses  were  combined  as 

e(i()  = sin2(x,)  + cos2(x<) 

where  x<  is  a linear  filter  response,  indexed  over  filter  position,  orientation, 
and  frequency  band. 
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4.4  Point  non-linearity 

Nachmias  and  Sansbury  (1974)  showed  that  the  results  of  a grating  contrast 
discrimination  experiment,  when  plotted  with  threshold  contrast  increment 
as  a function  of  the  base  contrast  from  which  the  increment  threshold  is 
being  measured,  produce  a dipper-shaped  curve.  These  authors  argued  that 
the  results  can  be  modelled  by  assuming  a sigmoid  non-linearity  following  a 
linear  detection  mechanism.  The  decision  mechanism  has  available  to  it  only 
the  output  of  this  non-linearity,  and  reliably  discriminates  between  inputs  of 
two  different  contrasts  when  the  difference  in  outputs  is  greater  than  some 
threshold. 

To  quantitatively  model  the  dipper-shaped  contrast  discrimination  curve, 
we  follow  Legge  and  Foley  (1980)  in  using  a non-linear  transducer  of  the  form 


T(Li)  = 


H£,|n 

|A-P  + s» 


where  T is  the  non-linear  transducer  output,  I,  is  the  linear  filter  response 
(indexed  as  above),  r is  an  overall  gain-setting  parameter,  n is  a real  number 
greater  than  2 (2.4  here),  and  s is  a semi-saturation  constant  (0.0075).  Given 
that  our  energy  calculation  described  above  already  squares  the  linear  filter 
resonses,  the  transducer  output  can  be  expressed  as 


T(ex) 


re 


n-2 


e,-  + s 2 


where  e,  is  short  for  e(x,)  in  the  energy  expression  above. 


4.5  Decision  stage 

4.5.1  Distance  calculation 

We  are  assuming  no  summation  among  transducer  outputs,  and  a decision 
mechanism  in  which  discrimination  is  governed  by  the  pathway  whose  trans- 
ducer output  shows  the  maxmimum  change  between  the  two  stimulus  presen- 
tations. Although  these  assumptions  are  probably  not  correct  in  detail,  they 
dramatically  simplify  model  theory  and  calculations,  and  thus  are  useful  as 
a first  pass  at  the  truth. 
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One  way  of  expressing  this  maximum  change  assumption  is  that  the  dis- 
criminability  between  two  images  is  assumed  to  be  proportional  to  a non- 
Euclidean  distance  between  the  points  representing  the  two  images  m the 
space  of  transducer  outputs.  That  is, 


D{sus2 ) = 


« £[r(e,(s,))  " r(e,(52))]9 

A i«  1 


where  s\  and  s2  axe  the  two  images,  and  n is  the  number  of  different  trans- 
ducer channels.  If  Q = 2,  this  expression  returns  the  Euclidean  distance 
between  points  in  the  transducer  output  space.  As  Q -►  oo,  the  distance 
metric  gets  closer  and  closer  to  the  maximum  change  model  described  above. 


4.5.2  Distance  to  probability 

This  distance  measure  can  be  used  in  the  following  generalization  of  the  Nach- 
mias  and  Sansbury  model:  Two  images  are  reliably  discriminated  whenever 
the  result  of  the  distance  calculation  for  the  two  images  is  greater  than  some 

threshold.  , . , ,. 

This  simple  generalization  is  sufficient  to  model  results  in  which  discrim- 
ination thresholds  are  measured  as  a function  of  a change  in  a stimulus  pa- 
rameter (e.g.  contrast,  as  in  the  Nachmias  and  Sansbury  results,  or  spatial 
frequency,  as  in  the  edge  transition  results  to  be  discussed  below.)  For  these 
tasks,  the  threshold  distance  can  be  understood  as  the  distance  which  gives 
a 75%  probability  of  discrimination  among  the  two  stimuli.  However,  for 
the  cockpit  display  visibility  modelling,  the  relevant  performance  measure  is 
the  probability  of  discrimination  given  two  images,  not  the  required  change 
between  two  images  given  a fixed  probability.  Therefore  a mapping  between 
distance  and  probability  is  required,  not  for  just  a single  value  of  distance 

and  probability,  but  along  the  entire  range  of  each.  . 

We  have  generated  this  complete  mapping  from  distance  to  probability 
using  two  sets  of  data:  Contrast  detection  psychometric  functions  from  Foley 
and  Leg ge  (1981)  and  contrast  discrimination  functions  from  Legge  and  Foley 
(1980).  In  the  former  data  set,  probability  of  detecting  a sine  grating  is 
plotted  against  the  contrast  of  that  grating.  Foley  and  Legge  showed  that 
these  data  are  well  fit  by  an  expression 

p(C)  = 100-50exp(-aC‘) 
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where  C is  contrast,  and  a and  b are  parameters,  fitted  to  one  observer  as 
a = 52.60  and  b = 3.0. 

For  the  latter  data  set,  a good  fit  is  obtained  using  the  transducer  function 
described  above.  This  transducer  expresses  distance  as  a function  of  linear 
filter  output,  but  can  be  expressed  equally  well  as  a function  of  grating  con- 
trast, since  linear  filter  output  is  proportional  to  contrast.  This  we 

can  express  both  distance  (i.e.,  transducer  output)  and  probability  as  a func- 
tion of  contrast,  and  have  only  to  invert  the  transducer  function  to  obtain  an 
expression  for  probability  as  a function  of  distance.  We  were  not  able  to  solve 
this  problem  analytically,  but  instead  generated  an  accurate  computational 
solution  that  was  incorporated  into  the  visibility  model  software  as  a lookup 
table. 

To  test  this  mapping,  we  applied  the  model  to  predict  psychometric  func- 
tions for  contrast  discrimination,  also  published  in  Legge  and  Foley  (1981). 
In  these  functions,  probability  for  detecting  a change  in  contrast  of  a sine 
grating  at  threshold  is  plotted  against  that  change  in  contrast.  The  model 
predicted  these  functions  very  accurately. 


4.6  Position  uncertainty 

One  important  task  for  performance  measurement  in  a cockpit  display  envi- 
ronment is  character  discrimination  as  a function  of  eccentricity.  However, 
predictions  on  0 vs.  Q discrimination  from  the  model  as  described  so  far 
incorrectly  assert  near  perfect  discrimination  out  to  as  much  as  16  degrees, 
whereas  in  reality,  probability  of  successful  discrimination  among  these  two 
characters  falls  to  chance  by  about  5 degrees. 

Because  of  the  lack  of  spatial  pooling  in  the  model,  the  task  of  discrimi- 
nating  between  0 and  Q becomes,  for  the  model,  the  task  of  simply  detecting 
the  diagonal  segment  in  the  Q,  a task  which  could  in  fact  be  performed  re- 
liably out  to  16  degrees.  But  for  letter  discrimination,  it  is  not  sufficient 
to  simply  detect  all  the  component  features;  accurate  spatial  relationships 
among  these  features  must  also  be  recovered.  This  led  us  to  consider  other 
psychophysical  tasks  for  which  accurate  localization  is  important,  most  no- 
tably the  three  dot  bisection  task  of  Yap,  Levi  and  Klein  (1987;  JOSA,  4, 
8,  pp.  1554-1561).  Here,  as  in  contrast  detection,  a linear  scaling  of  image 
area  with  eccentricity  leads  to  constant  performance  across  the  visual  field. 
However,  the  scaling  factor  for  three  dot  bisection  is  larger  by  about  a factor 
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of  three  than  the  scaling  factor  for  contrast  detection. 

The  fact  that  tasks  requiring  accurate  localization  scale  by  a larger  fac- 
tor than  that  of  contrast  sensitivity  suggested  to  us  the  need  for  a stage  of 
eccentricity  dependent  spatial  pooling  of  sensor  responses  following  the  lin- 
ear filtering  stage.  If  filter  responses  were  pooled  over  a progressively  larger 
area  towards  the  periphery,  then  the  central  visual  system  would  become 
increasingly  uncertain  of  the  spatial  position  of  a given  image  feature.  Fur- 
thermore, if  the  gain  of  the  pooling  filters  were  scaled  so  that  the  magnitude 
of  the  pooled  response  of  a uniform  input  were  unchanged  with  the  size  of 
the  input  area,  then  the  model  would  continue  to  accurately  predict  contrast 
sensitivity  in  the  periphery. 

When  we  incorporated  these  changes  into  the  model,  it  was  able  to  quali- 
tatively predict  published  results  in  letter  discrimination,  three-dot  bisection, 
and  contrast  sensitivity  in  the  periphery.  After  making  these  changes,  we 
found  further  verification  for  our  letter  discrimination  predictions  in  a recent 
report  by  Farrell  and  Desmarais  (JOSA  (1990)  7,  1,  pp.  152-159). 

4.7  Output  format 

Model  predictions  are  generated  in  files  for  which  all  parameters  are  fixed, 
except  for  eccentricity  (i.e.,  degrees  of  visual  angle  off  of  the  fixation  point.) 
The  output  files  thus  contain  a simple  two  column  table  of  eccentricity  - 
probability  pairs,  one  such  file  for  each  combination  of  image  pair,  lighting, 
and  fixation  depth  parameters. 

The  initial  set  of  model  results  included  predictions  for  two  different  fonts 
of  Q and  0 (tvpes  a and  b from  the  Apache  Longbow  Crew  Systems  Inter- 
face Document).  A screen  depth  of  30”  was  assumed,  coupled  with  four 
different  fixation  depths:  15”,  30”,  60”,  and  oo.  For  each  of  the  two  fonts 
and  four  fixation  depths,  we  generated  model  predictions  at  the  following 
luminance/illuminance  combinations: 


luminance 

illuminance 

10,000  fed 

350.0  fL 

10,000  fed 

100.0  fL 

10,000  fed 

31.6  fL 

10,000  fed 

10.0  fL 

1,000  fed 

10.0  fL 

100  fed 

10.0  fL 

10  fed 

10.0  fL 

These  illuminances  range  from  bright  sunlight  (10,000  fed)  to  late  dusk 
(10  fed).  The  luminances  range  from  the  maximum  available,  as  listed  in  the 
Apache  Longbow  document  (350  fL)  to  the  minimum  daylight  mode  setting 

^ 1U  I Li  j, 

Figures  2,  3,  4,  and  5 show  model  results  for  the  range  of  character  sizes 
and  lighting  parameters  described  above.  In  all  thse  figures,  fixation  depth 
is  equal  to  stimulus  depth  (30”). 

In  order  to  generate  complete  discrimination  contours  from  the  model 
output  files,  the  package  of  model  software  also  contains  a routine  which 
generates  probability  contours  from  55%  to  95%,  and  the  99%  contour.  This 
routine  performs  a linear  interpolation  on  the  probability  values  listed  in 
the  model  output  file,  to  determine  the  eccentricity  at  which  each  contour’s 
probability  value  would  occur.  It  then  computes  a set  of  x,y  points  (in  degrees 
of  visual  angle)  for  a complete  circle  at  that  eccentricity.  Additional  data 
may,  in  the  future,  require  refinement  of  this  module  to  produce  contours 
which  deviate  from  a purely  circular  shape. 


5 Validation 

5.1  Edge  transition  data 

The  discrimination  model  has  been  successfully  tested  against  the  Carlson 
and  Cohen  (197S)  edge  transition  data,  with  a fit  better  than  that  of  Carlson 
and  Cohen’s  original  (JND)  model. 

In  the  edge  transition  task,  observers  are  asked  to  discriminate  a change 
in  sharpness  of  an  intensity  edge,  as  a function  of  the  base  sharpness  of  that 
edge.  More  specifically,  in  the  Carlson  and  Cohen  data,  the  one-dimensional 
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percent  correct  discrimination 


Task:  Q vs.  0 (size  a) 

Luminance:  10.00  fL 

Illuminance: 

10  fed 

100  fed 

1,000  fed 

10,000  fed 


6 8 10  12 

eccentricity  (degrees) 


Figure  2:  Model  predictions:  Q vs.  0,  size  a,  luminance  fixed  at  10  fL,  four 
different  illuminances. 
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percent  correct  discrimination 


Figure  3:  Model  predictions: 
four  different  luminances. 


Q vs.  0,  size  a,  illuminance  fixed  at  10,000  fed, 
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Task:  Q vs.  0 (size  b) 

Luminance:  10.00  fL 

Illuminance: 

. 10  fed 

100  fed 

1,000  fed 

10,000  fed 


\ \ 


\ \ 
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eccentricity  (degrees) 


x-  n vc  O size  b luminance  fixed  at  10  fL,  four 
Figure  4:  Model  predictions:  Q vs.  O,  siz  , 

different  illuminances. 


4 6 8 10  12  14 

eccentricity  (degrees) 


Figure  5:  Model  predictions: 
four  different  luminances. 


Q vs.  O,  size  b,  illuminance  fixed  at  10,000  fed. 
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horizontal  cross-section  of  a vertical  edge  is  generated  as  an  error  function 
(erf),  where 

erf(x)  = (2/ v/jr)  jf  exp(— t7)dt 

The  sharpness  of  the  edge  is  controlled  in  the  following  parameterization 
for  x in  the  above  expression: 

x = dirfjyj log(2) 

Here,  d indexes  the  distance  across  the  edge  image  in  degrees,  and  ranges 
from  —wf 2 to  w/ 2 (where  w is  the  width  of  the  image  in  degrees),  and 
fe  is  the  frequency  at  which  the  modulation  transfer  function  for  the  edge 
has  fallen  to  one  half  of  its  maximum.  This  parameter  fe  thus  governs  the 
sharpness  of  the  edge,  with  higher  values  giving  sharper  edges. 

Given-  this  parameterization,  the  psychophysical  task  can  be  expressed 
as  follows:  From  an  edge  with  a fixed  fc  value  (call  it  /*),  how  much  does 
the  fc  have  to  increase  (call  the  increase  dfx)  so  that  the  change  in  edge 
sharpness  is  just  noticeable.  The  results  were  plotted  by  Carlson  and  Cohen 
with  log(/,)  on  the  abscissa,  and  log (<*/,/(/,  + dfx))  on  the  ordinate.  In 
general,  these  results  follow  a u shape,  with  the  minimum  near  a frequency 
of  1 cycle/degree. 

Figure  6 shows  the  edge  transition  data  from  Carlson  and  Cohen,  together 
with  their  model  fit  and  the  fit  of  the  discriminability  model  described  here. 
The  discriminability  model  was  fit  by  finding  the  value  of  dfx  at  each  fx 
which  gave  75%  probability  of  discrimination  between  the  two  edges. 


5.2  Character  discriminability  data 

We  have  begun  collecting  additional  data  to  test  the  model  s predictions 
in  a character  discrimination  task.  All  experiments  were  performed  on  the 
MacIIx.  The  stimuli,  as  in  the  model  runs,  were  uppercase  0 and  Q,  adapted 
from  the  Mac  Helvetical4  font  to  accurately  replicate  the  size  and  bitmap 
pattern  of  the  Size  A characters  described  in  the  Apache  Longbow  Crew  Sys- 
tems Interface  Document.  The  screen  luminance  was  variable  from  between 
0 and  13.5  ftL.  The  subject’s  distance  from  the  screen  is  30  inches.  Veiling 
illuminance  for  the  data  collected  so  far  was  approximately  10  fed. 
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Figure  6:  Edge  transition:  data  and  model.  Upper  curve  shows  the  predic- 
tions of  the  visibility  model  described  in  this  report;  lower  curve  shows  the 
predictions  of  the  JND  model  of  Carlson  and  Cohen. 
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A small  fixation  mark  (+)  was  continuously  present  at  the  center  of  the 
screen.  During  each  trial  a single  character,  0 or  Q,  was  Sashed  on  for  167 
mXat  an  eccentricity  e either  to  the  right  or  to  the  left  of  the  fb.at.on  pomjt 
This  direction,  as  well  as  the  identity  of  the  character  r^domiy  r^ed 
from  trial  to  trial.  The  eccentricity  e remaned 

as  did  the  luminance  of  the  characters  and  background.  The  subject  s task 
was  to  identify,  with  a keypress,  which  of  the  two  characters  was : *T^ 

A session  ended  when  each  of  the  four  possible  combinations  of 
identity  and  position  has  been  presented  n times,  where  it  is  typically  50. 

Preliminary  data  from  this  paradigm  are  shown  in  Figure  7. 

In  this  figure,  the  three  connected  points  marked  bY  the  heavier  crosses 
show  the  data  for  one  subject,  under  lighting  conditions  of  10 fed 
and  6 fL  screen  luminance.  Also  shown  in  the  figure,  marked  by  fifed  squares, 
is  the  model  run  with  lighting  conditions  most  similar  to  those  of  the  da  a 
run-  i.e.,  10  fed,  10  fL.  As  shown  in  the  figure,  the  slope  of  the  falloff  m 
performance  with  eccentricity  is  in  good  agreement  with  model  preictio^ 
Not  surprisingly,  though,  there  is  a ”dc  shift”  between  model  and  data;  that 
is,  the  eccentricities  at  which  these  reported  probabilities  occur  are  not  in 

eood  agreement  with  the  model  results.  , 

8 One  obvious  possible  reason  for  this  discrepancy  is  that  the  short  stimulus 
duration  used  in  the  experiment  (to  prevent  eye  movements)  reduces  the 
effective  contrast  of  the  stimuli,  thereby  significantly  affecting  performance. 
Since  the  model  as  it  now  stands  is  blind  to  changes  in  stimulus  duration, 
it  would  not  be  expected  to  accurately  predict  performance  for  arbitrarily 

short  duration  stimuli.  . . 

To  make  sure  that  an  increase  in  effective  contrast  does  increase  human 

performance  (i.e.,  to  make  sure  that  the  human  results  reported  above  are 
not  alreadv  at  a performance  asymptote),  we  used  a second  screen  lumi- 
nance configuration,  chosen  to  maximize  the  effective  contras  ^labk  on 
the  MacIIx.  In  this  condition,  the  characters  are  dark  on  a 13  5 & (max- 
imum possible  luminance)  background.  Results  axe  shown  in  the  figure j* 
empty  squares.  Under  these  conditions,  the  human  data  are  much  improved, 
shifting  the  performance  curve  outward  by  about  two  degrees.  .... 

Shown  also  in  the  figure,  for  comparison  purposes,  is  a mode!  run  in  which 
the  effective  contrast  is  low;  i.e.,  a 10,000  fed,  31.6  fL  condition.  Here,  the 
results  are  quite  similar  to  the  10  fed,  6 fL  data  run.  These  results  suggest 
that  some  additional  model  development  is  needed  to  accurately  measure 
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Percent  correct  discrimination 


Figure  7:  Letter  discrimination:  data  and  model. 
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the  effective  contrast  of  a stimulus,  given  its  time  course  and  other  relevant 
stimulus  and  observer  parameters. 


6 Future  directions 

In  general,  future  plans  for  this  project  involve  both  «rtencling  the  range 
of  model  applicability  and,  through  additional  data  coUection  and  model 
refinements,  improving  confidence  in  model  predictions.  Specific  plans  are 
outlined  below. 


6.1  Stimulus  parameters 

6.1.1  Change  and  motion 

An  important  area  for  extending  the  range  of  the  model  is  into  temporal 
stimulus  parameters;  i.e.,  explicitly  modeling™.  al  respond  toitaM 
are  moving  or  otherwise  changing  in  time.  The  importance  of  the  ™ 
made  clear  in  the  previous  section,  where  it  was  described  that  the  short  tune 
course  of  the  experimental  stimuli  may  be  changing  their  effective  contrast. 
Modeling  response  to  motion  would  also  vastly  increase  the  useful  range 
of  stimuli:  target  detection  could  be  modeled,  a.  weU  as  djm^cc^kpit 
display  events  such  as  flashing  lights  or  rapidly  deflecting  meter  needles. 

6.1.2  Color 

Extending  the  model  into  the  chromatic  domein  i.  teUtiedy  atraighdomard, 
end  useful  for  the  seme  reasons  cited  ebovefor  change  and  motion  detection. 


6.2  Tasks 

6.2.1  Discrimination 

The  model  as  it  now  stands  can  generate  predictions  for  discrimination  tasks 
other  than  character  discrimination.  Tasks  such  as  target  detection  and 
meter  needle  position  discrimination  can  be  investigated,  even  before  the 
model  is  extended  into  the  motion  or  color  domains. 
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6.2.2  Localization 

Another  important  task,  especially  for  target  detection,  is  stimulus  localiza- 
ion;  i.e.,  the  accuracy  with  which  stimulus  position  can  be  reported.  Because 
of  the  areal  summation  currently  built  into  the  model,  accurate  localization 
of  stimuli  will  decrease  as  a function  of  stimulus  eccentricity.  Additional 
research  will  be  required  to  determine  other  relevant  stimulus,  task,  and 
observer  parameters  affecting  performance  in  this  task. 

6*3  Performance  measures 

6.3.1  Probability  and  confusion  matrices 

Currently,  the  model  generates  an  estimate  of  probability  correct  for  discrim- 

m unube!uVe€n  a/aiF  °f  Stimuli-  A useful  and  straighforward  generalization 
would  be  the  production  of  a full  confusion  matrix  for  members  of  a set  of 

allowable  responses.  This  performance  measure  would  become  especially  im- 
portant if  the  vision  model  were  given  the  responsibility  of  providing  to  other 
modules  the  identity  of  a percept,  rather  than  just  a measure  of  success  in  a 
simple  discrimination  task. 

6.3.2  Response  time 

Another  possible  performance  measure  is  the  time  required  to  perform  a 
vision-based  task.  The  usefulness  of  accurate  modeling  of  this  performance 
measure  depends  on  the  level  of  temporal  detail  required  by  other  MIDAS 
modules.  Since  visual  response  times  are  almost  always  in  the  100  to  500 
millisecond  range,  accurate  modelling  would  be  irrelevant  if  other  modules 
were  operating  in,  for  example,  the  5 to  10  second  range.  Moreover,  modeline 

response  time  would  probably  require  significant  additional  effort  to  extend 
the  current  model. 


6.4  Observer  state  parameters 

6.4.1  Adaptation 


The  dynamics  of  light  adaptation 
entire  careers  have  been  devoted. 


is  a complex  research  question,  to  which 
However,  much  would  be  gained  from  a 
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simple  model  in  which  the  decay  parameter  for  recovery  from  a change  in 
lighting  depends  on  the  absolute  light  level  and  the  size  of  the  change.  More- 
over, modeling  can  also  be  simplified  if  the  time  scale  over  which  recovery  is 
modeled  is  restricted  to  a specific  range;  e.g.,  100  to  1000  msec.  However,  it 
should  be  noted  here  that  the  addition  of  light  adaptation  to  the  vision  model 
puts  additional  responsibilities  on  other  MIDAS  modules  as  well,  since  some 
module  would  be  required  to  provide  the  vision  model  with  ongoing  estimates 
of  lighting  conditions,  based  for  example,  on  sun  position,  cloud  cover,  and 
vehicle  position. 

6.4.2  Resource  loading 

Resource  loading  is  an  ill-defined  but  important  parameter  affecting  pilot 
performance.  One  way  to  start  investigating  this  parameter  is  to  consider 
performance  in  dual  task  situations,  about  which  a moderate  amount  of  re- 
sults has -.been  reported  (e.g.,  Pashler,  Cognitive  Psychology,  1989).  The 
general  strategy  here  is  to  consider  resource  loading  not  in  terms  of  a homo- 
geneous capacity  model,  but  in  terms  of  specific  interactions  among  specific 
kinds  of  tasks. 

6.5  Model  refinement 

6.5.1  Contrast  gain  control 

Recent  results  (Lubin,  ARVO,  1989-1990)  suggest  a model  stage  in  which 
outputs  of  different  spatio-temporal  frequency  channels  set  the  gain  of  neigh- 
boring channels  via  a division-like  operation.  These  data  are  relevant  to  the 
visibility  model  because  they  provide  evidence  against  the  simple  indepen- 
dence of  frequency  channels  in  the  current  model.  Inclusion  of  such  a gain- 
setting  stage  would  improve  model  performance  across  most  discrimination 

tasks. 

6.5.2  Detection  vs.  localization  scaling 

These  two  different  stages  of  scaling  were  already  discussed  in  the  model  de- 
scription section  above:  The  model  contains  an  earlier  stage  at  which  sensor 
size  scales  as  a function  of  eccentricity,  and  a later  stage  at  which  sensor  re- 
sponses are  pooled  as  a different  function  of  eccentricity.  For  simplicity,  this 
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second  stage  of  scaling  is  currently  implemented  as  a scaling  down  of  the  dis- 
tance in  the  transducer  output  space.  However,  this  Bmplifintinn  does  not 
always  accurately  model  the  presumed  scaling  operation,  since  it  decreases 
the  output  distance  even  between  spatially  distant  pairs  erf  stimuli,  for  which 
no  pooling  would  be  expected.  An  implementation  more  isomorphic  with  the 
presumed  operation  is  thus  required. 

6.6  Data  collection 

6.6.1  Model  verification 

Data  collection  is  ongoing.  Current  plans  include  extending  the  range  of  char- 
acter discrimination  data  to  other  lighting  conditions  and  character  pairs. 
Performance  in  other  cockpit  display  tasks,  such  as  detecting  a change  in 
meter  needle  position,  will  also  be  measured.  Moreover,  as  the  model  is 
extended  to  other  performance  measures  and  stimulus  and  observer  parame- 
ters, additional  data  collection  will  become  necessary.  Tasks  such  as  moving 
target  detection,  with  performance  measures  such  as  the  response  time  to 
accurately  localize  the  target,  may  be  undertaken. 

6.6.2  Normative  data 

Another  useful  set  of  data  would  be  normative  data;  that  is,  data  for  a wide 
range  of  individuals  within  the  relevant  subject  population,  so  that  variance 
estimates  for  different  performance  measures  could  be  obtained.  Such  es- 
timates would  help  establish  more  accurately  the  probability  of  success  in 
different  mission  scenarios. 
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APPENDIX  C 


LIGHTHOUSE  VP  SOURCE  CODE  DOCUMENTATION 
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1 Organization  Of  The  Source  Code 


1.1  Th#  VP  Executable  Program 


All  of  the  source  code,  with  the  exception  of  main  , c,  is  used  to  generate  a number  of  libraries.  When 
main . c is  compiled  with  these  libraries,  the  VP  executable  program  is  produced. 

Environment  Variables 

There  are  several  environment  variables  that  need  to  be  set  in  order  to  properly  compile  and  run  VP.  These 
wiables  should  be  set  by  running  the  command  file  . vpenv  in  the  VP  root  directory.  The  first  step, 
though,  is  to  edit  . vpenv  so  that  the  environment  variable  VP  is  going  to  be  set  to  the  VP  root  directory 
For  example,  if  the  VP  directory  hierarchy  is  rooted  in  /usr/f oo,  edit  the  line  that  beams  "■•tanv 


aetenv  VP  /usr/foo/vp 

Once  this  is  done,  the  rest  of  the  program’s  environment  variables  can  be  set  by  executing  . vpenv  with 
the  UNIX  source  command. 


1.2  The  Source  Code 

With  the  exception  of  the  main  program  (vp/erc/main . c)  all  of  the  source  code  is  used  to  make  the  VP 
library  files.  Each  subdirectory  of  vp/src  contains  the  files  that  are  used  to  make  one  library  of  routines. 
For  example,  the  source  files  in  vp/arc/model  are  used  to  make  the  library  vp/lib/libmodal  a 
All  of  the  header  files  are  kept  in  vp/lib. 

The  Main  Program:  vp/arc/main . c 

The  function  of  the  main  program  vp/sre/main . c is  straightforward.  First  it  calls  initialize 
(which  opens  and  initializes  all  of  the  initial  windows)  and  then  it  calls  menu_loop  (which  processes 
intermediate  events  while  waiting  for  a menu  event). 

The  vpmain  Library 

The  routines  in  the  vpmain  library  take  care  of  much  of  the  overhead  in  the  program.  Start-up 
iiuuahzauon,  window  maintenance,  message  and  error  handling,  event  handling  and  list  operations  are 
implemented  in  libvpmain.a.  The  C files  described  in  this  section  make  libvpmain.a  and  are 
found  in  vp/sre/vpmain. 

event . c Contains  the  function  proces3_events,  which  wails  for  an  event  to  queue  and 
processes  it  accordingly.  It  returns  a structure  of  type  Event. 

gr  id . c — Contains  the  routine  that  draws  the  scale  grid. 

init . c — Opens  the  start-up  windows  and  initializes  the  eyes. 

list . c — Functions  for  operations  on  two-way  open-ended  and  circular  linked  lists  and  ftarh 

menu . c — The  menus  for  the  message  window  and  environment  windows  are  defined  here.  (Note 
that  all  environment  windows  have  the  same  menu.) 

message. c — Functions  for  displaying  text  in  the  message  window,  maintaining  the  message 
suck  and  getting  input  text. 
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view . c — Contains  the  routines  thal  change  the  view  in  environment  windows. 

window . c — Function  thil  open  tnd  dose  the  menage  tnd  orriroo«ie«  window!  and  find  dan 
in  the  window  list 


The  / n k / 1 ihmod«  1 a)  crovidcs  the  routines  for  aeiting,  maintaining  md  drawing 

new  locations  and  delete  them  from  the  environment,  and  the  lighting  routines. 

create . c — Contains  the  routines  that  "create"  models,  which  essentially  means  reading  the 
model  file  and  adding  the  model’s  data  structure  to  the  model  tree. 

draw . c - Contains  the  routines  that  draw  the  model  hierarchy  and  individual  models. 

edit . c — Contains  the  routines  that  move  and  delete  models. 

lexer . c — The  lexical  analyzer  used  by  the  model  file  parser. 

light . c — The  routines  that  define  and  control  lighting. 


pa  rse . c — The  model  file  parser. 

pic* . c The  routines  used  to  "pick"  a model  that  is  drawn  on  the  screen. 

symbol . c — Symbol  table  support  for  the  parser. 

tree . c - Tree  traversal  routines  that  apply  a function  (specified  as  an  argument  to  the  traversal 
function)  to  each  node  in  the  tree. 


utils . c — Various  general  utilities. 


The  ^ctorlUnjy  ^ contains  routines  for  vector  and  matrix  operations.  (Some 

vect^o^ration^areperformed  with  macro  substitution  and  are  defined  in  vp/  include  / vector . h.) 


mat  r ix . c — Routines  that  perform  operations  on  homogeneous  transformation  matnees. 


vector . c — Routines  that  perform  vector  operations. 


Hie  eye  library  (vp/lib/libeye . a)  contains  the  all  of  the  routines  that  at 
and  vision. *lt  is  important  to  note  that  even  though  retina  window  routines  are 
they  must  use  the  window  support  that  is  provided  in  libvpmain . a. 


associated  with  the  eyes 
defined  in  libeye  • a. 


p c Contains  initialization  and  fixation  routines. 

menu . c - Defines  the  menu  for  the  retina  window.  Note  that  some  eye  operations  are  executed 
from  the  environment  window  menu. 


ret  ina . c — Routines  that  create,  draw  and  delete  retinal  objects. 


window . c — Routines  for  opening  and  drawing  retina  windows. 
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2 Lists,  Stacks  and  Trees 


‘rC  dC,fnCd  “ li3t’h  “d  by  functions  in  list.c.  These  date 

smictures  are  dynamicaUy  implemented  so  that  there  are  no  predefined  limits  on  their 

2.1  UttS 


A generic  list  node  is  defined  in  list. has 

struct  list_node  { 

void  *dat; 

struct  list_node  *prev; 
struct  list  node  *next; 

) ; 


pus  structure  is  the  basis  of  the  program’s  open-ended  lists,  circular  lists  and  sacks  The  fields 
prev  and  next  point  to  the  previous  node  and  the  next  node,  respectively  The  use  of  the  nnv 
pointer  allows  for  two-way  operation.  The  data  field,  dat.  bVS  tTvTiflLJL?! 
void  pointer  as  a data  field  is  what  gives  the  list  node  its  generic  quality  Anv  noin  ter 
converted  to  void  through  a cast  and  then  converted  back  to  its  rigtai 

*-«*-—*»-*.  ^(^ftotheh***. 

void  list_append (List  ‘list,  void  *data) ; 

Boolean  delete_listnode (List  *list,  void  *data) • 
void  delete_list (List  *list); 

The  argument  list  points  to  the  head  of  the  list,  data  points  to  the  data  that  is  to  be  added  to 

?,d'le“d  ***»  li.t_.pp.nd  is  used  to  append  ml££Z  St" 

list,  or  to  create  a new  list  when  list  is  equal  to  NULL,  delete  listnode  remove  ih»  i:« 
node  containing  data  from  the  fist  It  returns  false  if  the  list  wasempty  (equal  to  NULL)  <r  if 
data  wasn’t  found  in  the  list  delete_list  deletes  an  entire  li^edm  ^ 

Circular  lists  are  implemented  in  very  much  the  same  way.  except  they  don’t  have  a beginning  or  an 
end.  The  prototypes  for  the  circular  list  support  functions  are  shown  below.  PIBU  * 

Boolean  insert_circlelistnode (List  *list,  void  -data) ; 

Boolean  remove_circlelistnode (List  ‘list,  void  *data>; 

insert  circlelistnode  is  used  to  add  data  to  the  list  A new  list  node  for  data  is  created. 

and  « is  inserted  into  the  list  immediately  after  the  node  p£mej  J Sy  TS? 

:1" ‘‘UMnod.  defew  the  node  confining  <ut.  (tom  ft.  lot  If  the  to  £ 
empty  or  if  data  wasn  t found,  remove_circlelistnode  returns  FALSE. 

2.2  Slacks 

°f  a ywktd  iist  in  wwch  **  «*  °{ **  «« « **  wp  of  the 

stack.  Type  Stack  is  defined  in  list  .h  to  be  exactly  the  same  as  List  — a pointer  to 

listed*  below1  St-n°de  prototyp“  of  lhc  functl0ns  are  used  to  push  and  pop  thenack  are 
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void  push (Stack  *atack,  void  *data> ; 
void  *pop (Stack  *stack) ; 


push  creates  a stack  node  for  data  and  pushes  it  onto 

push  appends  the  dau  to  the  end  of  the  list  and  sets  stack  to  jxmm  to 

the  new  top  of  the  stack.  To  pop  the  stack  pop  assigns  the  data  m the  last  lm  node  to  a teatpoiity 
the  former  top  of  stack  (which  was  saved  in  a temporary  variable). 


A generic  tree  node  is  defined  in  vp/ include/list  .h  as  follows. 

typedef  struct  tree_node  *Tree; 
struct  tree_node  { 


void 

*dat; 

Tree 

*prev; 

int 

numlinka ; 

Tree 

*link; 
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3 VP  Windows 


Each  windowhas  a number  of  parameters  that  mua  be  known  by  the  program  in  crier  for  it  to  be 
“*wn  P*operiy.  Information  about  the  projection  and  view  transformations,  for  The 

program  must  also  know  exactly  what  is  to  be  drawn  in  the  window.  This  section  deals  with  the 
issues  of  window  data  and  the  implementation  of  drawing  the  windows. 

3.1  Window  Organization 


Each  window  that  is  opened  by  the  program  has  a window  structure  associated  with  it.  The 
Window  structure  (shown  below)  is  defined  in  vp.h. 


struct  window  { 

long  wid; 

V 

int  rosnuid; 
char  *titla; 

bar.  */ 

void  <*draw_wintype) 

window  */ 

float  bkgnd [ 4 ] ; 

Xfrm  cop; 

view.  */ 
float  vrp  [ 3 ] ; 

Angle  fovy; 
float  aspect; 

Model  *vcam; 

Tree  modeltree; 

*/ 


/*  Graphical  window  id. 

/*  Tha  window's  menu  */ 
/*  Displayed  in  title 

(Window  *w) ; /*  Draws  the 

/*  Background  color.  */ 
/*  Center  of  projection  of 

/*  View  reference  point.  */ 

/*  Vertical  field  of  view  */ 

I*  x:y  aspect.  */ 

/*  Camera  model  at  cop.  */ 

/*  Tree  of  3D  object  models. 


List  objects; 

struct  ( 

unsigned  int 
unsigned  int 
unsigned  int 
} status; 

}; 


/*  Generic  list.  */ 

/ * Various  flags.  */ 
wintype  : 2; 
is_main  : 1; 
is  lit  : 1; 


typedef  struct  window  Window; 


Some  of  window  s members  apply  to  every  window,  while  others  are  only  by  certain 

types  of  window.  Below  is  a short  description  of  the  structure’s  members. 

wid  — The  graphical  window  identifier. 


menuid  — Each  window  uses  its  own  menu,  and  this  is  the  identifier  of  its  menu.  This 
helps  to  eliminate  the  need  for  a single  massive  menu. 

title  — The  title  that  is  displayed  in  the  window  frame’s  title  bar. 

^raw— 1 tintype  Points  to  the  function  that  draws  the  type  of  window  that  is  specified  in 

status . wintype.  Each  type  of  window  is  drawn  by  a specific  function.  At  the  time 
that  the  window  is  being  opened,  the  appropriate  drawing  function  is  assigned  to 
draw_wintype. 
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bkgnd  — The  background  color  of  the  window. 

cop  — A transformation  matrix  that  specifies  the  window’*  center  of  projection  and 
describes  the  view.  Used  only  by  three-dimensional  windows. 

vrp A vector  specifying  the  view  reference  point  of  a three-dimensional  window. 

fovy  — Specifies  the  vertical  field  of  view  for  perspective  projection  in  a three- 
dimensional  window. 

aspect  — Specifies  the  field  of  view  m the  horizontal  direction,  ft  is  equal  to  the  x.7 
aspect  ratio  of  the  window’s  dimensions  in  screen  coordinates. 

vcam  — A camera  icon  that  is  drawn  at  the  window’s  cop.  Ibis  can  only  be  seen  from 

other  environment  windows,  and  acts  as  an  aid  in  relating  10  the  view. 

mode  It  re e _ The  tree  of  graphical  objects  that  are  part  of  a window’s  three-dimensional 
environment 


objects  — A list  that  can  be  used  for  any  purpose,  ft  has  no  dedicated  use. 

status  — A set  of  status  flags,  status . wintype  denotes  the  -type"  of  window  — 
THREE_D  for  an  environment  window,  retina  for  a retina  window,  or  MESSAGE  for 
•k»  m#>cc aaft  window. 


^0*  message  window,  tee  is  no p~le«mi«<l  limi. on  te  numtoof 
windows  that  can  be  opened  in  the  program.  The  user  can  open  as  many environment .and  retma 
windows  as  practicality  (and  the  computer’s  memory)  will  allow.  VP  ktepsmck  of aU  ofti 
ZmZs  by  teeping  Ihem  all  in  a single  list  ft  opened  properly.  «ch  window  vnD  tavea 
window  structure  in  the  global  window  list  winList.  WinList,  declared  m vp.  WP® 
Circular  list  that  contains  pointers  to  each  window’s  Window  structure.  The  order  of 
the  windows  in  the  list  is  meaningless.  The  only  requirement  is  thai 1 each 'existing  window  appear 
? List  exactly  once.  As  Endows  are  opened,  they  are  added  to  the  list.  Ukewtse,  as  they 

are  closed  they  are  deleted  from  the  list. 

« vp  hss  s specific  ^dow^nin, 

Window  pointer  to  a basic  window  that  is  ready  to  be  customized.  Its  prototype  ts  listed  below. 

Window  *open_window (char  ‘title,  int  corners [4] ); 

The  first  parameter  specifies  the  tide  that  is  to  be  displayed  in  the  title  to  of  the  window.  The 
second  parameter  is  an  array  that  contains  the  coordinates  of  the  window  s borders  °" 

The  fim  two  elements  of  the  array  art  the  screen  x values  of  the  window  s comers,  while  the  thud 
and  forth  elements  are  the  screen  y values  of  the  window  s comers. 

o-en  window  starts  by  allocating  memory  for  a Window  structure,  and  then  adds  it  to  die  global 
it  opens  a window,  configures  the  graphics  - die  display  mode  is  configured  to 
RGB  double  buffer  mode  M and  queues  window  maintenance  and  mouse  devices. 
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To  draw  a list  other  than  WinList  in  which  the  order  of  drawing  is  not  a cooridenthm.  use 
draw_other_windows  as  shown  below.  «*«wh»ioii,  use 

draw_othsr_windows (NULL,  subset) ; 

This  line  of  code  would  draw  every  window  in  the  list  subset  whose  windnu  amn.  _ - 

is  not  equal  to  null.  Of  course,  there  should  never  be  a NULL  entry  hi  a window  list 
3.2  The  Massage  Window 

TTie  message  window  is  dedicated  to  displaying  text  — primarily  inymy^f  mnA 

(At  this  tune,  the  message  window  is  limited  to  displaying  a single  line  of  text)Only  ooeSSE 

window  is  opened  and  it  cannot  be  closed.  uw  w iext;  unty  one  message 

•Hie  message  window  is  opened  in  initialize  with  a call  to  opan  aasaao. 
which  is  defined  m vpxnain/window.c.  Its  prototype  is  listed  below.  ~ *** 

Window  *open_inessage_window(char  -title,  int  corners (4]), 

The  window  is  tall  enough  to  accommodate  a line  of  text,  and  as  wide  as  the  disnlav  *»«.  ik. 
dmwtng  uncuon  that  gets  assigned  to  die  draw.wityp. 

Window  structure  is  draw_message_window.  ™«**age  wmoow  a 

The  Message  Slack 

TTte  message  window  uses  a stack  to  maintain  text  that  it  needs  to  “remember"  The  stack  is 
declared  in  vpmain/mesaage.c as  follows:  * 

static  Stack  mag_atack; 

draw  me«age  window  simply  draws  the  text  that  is  at  the  top  of  the  stack.  New  taxi  i. 
displayed  by  pushing  u onto  msg_3tack.  Similarly,  popping  thewck  wfll 
W.4  the  previous  rest  The  fuueiions  used  lb,  'reSJ  L 

vpma  i n /me  3 s a ge . c ; their  prototypes  are  listd  below.  denned  m 

void  push_message (char  *nawmsg) ; 
char  *pop_message(void) ; 

Bod!  Of  the  functions  call  draw_meS3age_window  to  display  the  updated  message 

pop_message  returns  a pointer  to  the  popped  text.  “puawa  message. 

3.3  The  Environment  Window 

Environment  windows  display  a twodimensional  rendering  of  three-dimensional  obiect  data. 
Three-dimensional  graphical  objects  are  stored  in  constructs  called  Models,  and  collectively  tbev 
ajmpose  the  program  s graphical  object  environment  The  models  are  stored  in  a singfe  oee 
structure  that  is  shared  by  all  of  the  environment  windows.  Thus,  each  *»!«;,,  euviramnent 
window  will  render  the  same  group  of  models.  The  only  difference  between  dXn 
windows  ,s  die  view  from  which  the  models  are  beeing  seen.  The  program  pta^  SSS2 
on  the  number  of  environment  windows  that  can  be  open  simultaneously.  P 1 

Opening  An  Environment  Window 

^Ten,Winr"redwi,tl“““  ■>P--*».lr<.™»nt_.lnao..  Hrefreredoo 
Seefinedm  vprain/wmdow.c;  us  protype  is  listed  below. 
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rtoen  window  also  initializes  some  of  the  members  of  the  new  window  stroctnic  the  wfadow 

E&ZSTty  -i»op.n)  i.  » <he  .Id  Md.  *. 

*^t)  is  assigned  to  the  title  field,  and  the  modeltre.  and  ob  j.ctsfieUbamsm 
JETS  HUL^Tp«n_window  ends  by  returning  a pointer  to  the  new  window  snetme. 

has  a corresponding  function  that  draws  that  type  of  window.  As  an  example, 

the  lines 

draw_ ret ina_window( retwin)  ; 
swapbuffers () ; 

would  draw  the  retina  window  associated  with  the  Windowstructure 

Often  though,  it  is  necessary  to  redraw  all  of  the  window,  without  evm  lmowtag  I 

« or  what  type  they  art.  This  can  be  done  by  calling  the  function  dr»w_all_windows.  Its 

prototype  is  shown  below. 

void  draw_all_windows (void) ; 

. ml1  windows  located  in  vp/src/vpmain/window.c,  draws  all  of  the  windows  to 

Z&S&S’EZtli  A *.  Sc  of  lte cw  wfa  »«do.  tod«  M 

iiLte  originalwin.  Hen  iioavcnw  Bi„Li,t.*.«.gMCh«»V>*»«l"‘»kne. 

WINDOW (WinList) ->draw_wintype (WINDOW (WinList) ) f 
swapbuffers  0 ; 

WINDOW  is  a macro  defined  in  vp.h  that  casts  a List  node’s  dat  field  to »a  Poi««^ 
Window.  So  for  each  Window  structure  in  the  global window  list 

the  window  drawing  function  specified  in  the  structure's  draw.wintyp- > fidi  Norn  aUof 
the  window  drawing  functions  must  have  the  same  parameter  — a pointer  to  "indow.  Alto  tte 
eminHist  has  been®  traversed,  die  current  graphics  window  is  reset  to  die  tdentifier  stored  m 
originalwin,  and  control  is  returned  to  the  calling  function. 

a k *r  crw'ifir  rvder  in  the  global  window  list,  and  the  node  to  which  tfinLiat  points 
As  there  is  no  specific  order  in  tne  gioi*u  " A A windows  is  uncertain, 

freouently  changes,  the  order  in  which  draw_all_ windows  draws  th  . 

SometimL  though,  it  is  desirable  to  consistendy  draw  the  same  window  firsu  as  is  ihe  «« 
interactively  moving  a model  or  setting  the  view  in  an  en^nment  wmetow r£ 
desirable  to  draw  . subset  of  the  global  wmdow  1m  i^nList.  The  luncuon 
draw_other_windows  serves  both  purposes.  Its  prototype  is  listed  below. 

void  draw_other_windows (Window  *win.  List  others) ; 

There  are  two  key  differences  between  this  function  and  dra w_a 1 l_windo ws • 

Stead  of  traversing  the  list  pointed  to  by  WinList.  it  traverses  the  tot  specified 

The  second  difference  is  that  it  draws  all  of  the  windows  in  the  others  list,  except  for  the 

window  specified  by  win.  Consider  die  code  listed  below. 

draw_environment window (envwin) ; 

swapbuffers  0 ; 

draw  other_ windows (envwin,  viewlist) ; 

In  this  example,  the  environment  window  specified  by  envwin  would  be  drawn  first  Then 
draw_other_windowa  would  draw  all  of  the  windows  m the  circular  tot  viewlist.  except 

for  the"window  specified  by  envwin. 
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Window  *open_environment_window (char  ‘title,  corners  [4]), 

ftlDCtKJII  ODCQS  the  window  with  • r«ti  fn 

transformations.)  Environment  window,  use  . perspeSS  — BodeI,in« 


Setting  The  View 
Theinida 


Drawing  The  Environment 

Environment  windows  are  drawn  bv  the  Amnim  ■* 

defined  in  vpmain/ window,  c.  Its* protype  is  lis^^^*DVir0iamnt--wind0K  wbkb  * 

void  draw_environment_window (Window  *U)  ; 

The  funSdon  st^by^m^ ** *"»• 
with  the  -color  specified  in  w^>bJcgnd.  The  z-buffer  is  also^d^/^!^^  W“dOW 
projection  and  dm  view  and  draws  the  tree  of  graphical  mode^sSor^  *“  * 

perspective (w->fovy,  w->aspect,  NEAR,  FAR) • 
inverthomomat (view. mat,  w->cop.mat) • ' 

loadmatrix (view. mat) ; 
scale_grid() ; 

draw_model_tree(w->modeltree) ; 

perspective  sets  the  projection  according  to  the  window’s  vertical  «mt  i.  • . - . 

view.  The  view  is  set  by  loading  the  inverse  tran«r««  rfT  and  horizontal  fields  of 

reference  grid  is  drawn  centered  at  the  world  oricin  uHih  cop  °nt0  lhe  matrix  stack.  Then  the 

3.4  The  Retina  Window 


/*•  under  reconsmiction  ••/ 
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4 Models 


A .hiecHligiensioial  graphical  cfclect  model  Ip  v>  I.  ■nedfa.lto 

Models  ,re  collectively  voted  in  t tree  nraenne  Ibil  u coouk*  toeiefa  si'rtrc»ni*otwinaow^TK 

^taelf  I.  cooipwed  of  . ok  of  o«  or  oWoa  primiom  A ^ 

segment  is  stored  in  a data  structure  of  type  Segment,  which  specifies  lists  . . 

thfpolygonal  faces  of  the  segment  Models  are  creattd  and  added  to  the  environment  through  a 
pvsCT  ftatreafe  model  description  files  that  conform  to  vP’smodel  description  gramnmr. 


The  Model  Description  Language 
The  model  description  language  that  VP 


employs  is  expressed  in  Backns-Naur  Form  in  Figure  4.1 


below. 


start  —*  modlist  tot 
modlist  -*  mdef  modlist 
I € 

mdef-*  model  id  TV*7"  seglist T 
seglist  —*  sdef  seglist 
I € 

sdef-*  segment  Id  T xfrm  prim  seglist  •}’ 
xfrm  — ► tmat 
I xiist 
le 

xiist  -*  translate  ’(’  Duni  V num  7 Bnm  T 
I rotate  ’(’  num  7 num  7 num  ’)’ 

I scale  ’(’  num  num  V num  ’)’  xiist 
le 

tmat  -*  tm  T T num  7 num  7 num 
num  7 V 

*(’  num  7 num  7 num  7 num  T7 

*(•  num  num  7 num  7 num  ’)*’,* 

•(*  num  num  ’,’  num  num  T)’V 

prim  —*  id  ’(’  T V 


id  -*  letter(leueridigit)* 
num  — » digit* 
letter  -*  [a-zA-ZJ 
digit  -*  [0-9] 


Fig.  4.1.  The  grammar  for  VP’s  model  description  language. 

The  grammar  allows  for  one  or  more  models  to  be  defined  in  a single  file.  Bach  model  dotation 
beginswith  the  keyword  model,  which  is  immediately  followed  by  the  name  that  is  to  be 
S model.  (See  the  grammar  production  for  mdef.)  Then 

location  and  orientation  can  be  specified,  and  finally  the  segment  tree  is  defined.  The  segment  tree 
is  a group  of  one  or  more  segment  definitions.  Each  segment  definition  begins  with  the  byword 
segSTn?  wtah  is  follow!?  by  die  segment’s  name.  Then  *•*»*"*  * ‘ 

muSformation,  and  the  primitive  is  specified.  The  pnmmve  is  essennaUy ' ^ 
that  contains  the  vertex  coordinates  and  face  vertex  lists.  A pnmmve  file  must  have  the  Mansion 
^eg  bTL ten  in  this  example,  the  extension'*  not  used  in  die  model  desenpuon  file.  The 

following  code  is  a definition  of  a cube  model. 
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model  cube  model 
( 


segment  cube_segment  { 
cube  () ; 

) 


cub#-fflod*1  “ created  at  the  woridcrigin.  It 
pucea  at  the  world  origin  because  the  identity  matrix  is  assigned  to  "»~Ht  and  their  mi3i  •» 

vertex  and  face  information  from  the  file  cube . seg,  which  is  shown  below.  ** 


111 
110 
10  0 
10  1 
0 0 1 
0 0 0 
0 10 
0 11 
* 

12  3 4 
3 € 5 4 
5 6 7 8 
18  7 2 
14  5 8 
2 7 6 3 


Each  line  before  the  first  asterisk  specifies  the  x.  y and  x coordinates  of  a vertex  Each  line  „f,,,  ,k- 


Transform auons  may  be  described  in  one  of  two  ways.  One  way  is  to  use  the  matrix  onentort 
ranalate,  rotate  and  scale.  The  alternative  is  to  directly  specify  the  mmnu.. 
transformation  matrix.  The  two  methods  are  exclusive  — they  be  combined.  (S  Jtat 

grammarproductionfor  xfrm)  The  following  example  is  will  serve  as  an  fllSSf' 


model  eyes 
( 

tra  ((0.7071,  0,  -0.7071,  0), 

(0,  1,  0,  0), 

(0.7071,  0,  0.7071,  0), 

(0,  10,  0,  1)); 


segment  laft_eye 
{ 

translate  (3.2,  0,  0); 
scale (0 . 44,  0.44,  0.44); 
sphere ( ) ; 


segment  cornea 
( 

translate (0,  0,  0.7); 
scale (0.25,  0.25,  0.25); 
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sphere () ; 


» 

) 

segment  right_eye 

translate (-3.2,  0,  0); 
scale (0.44,  0.44,  0.44); 
sphere () ; 


segment  cornea 


( 


) 


translate (0,  0,  0.7); 
scale (0.25,  0.25,  0.25); 
sphere ( ) ; 


1 


/*  model  eyes  */ 


The  eyes  model  describes  two  ’eyeballs’  whose  centers  are  6.4  cm.  apart 
the  top  of  the  model  description  indicates  that  the  given  mam  jdwuM  be  ^ “ ^ 
coordinate  transformation.  The  tm  matrix  above  wiU  place  the  model  eyes  10  cm.  above  the 
world  origin,  and  the  model  will  be  rotated  45  degrees  about  the  y axis. 

Each  eyeball  is  constructed  with  two  instances  of  the  sphere  0 priimdve.  (In  ^J*f^®*** 
segment  file  sphere. seg  describes  a sphere  that  is  5 cm.  m **»««•)  J1* 
l^t  eve  is  located  at  (3.2,  0.  0)  with  respect  to  the  ongin  of  the  model,  acaleu  used  to 
transform^ the  sphere  ( ) primitive  to  the  proper  size.  Sole  that  scale  has  immediately  local 
scope  - it  is  not  carried  through  the  hierarchy  as  translation  and  rotation  are.  !•* t_eye  hw 
the  nested  segment  cornea.  Each  nested  segment  is  a dependent  in 
the  hierarchy.  Therefore,  cornea  will  be  transformed  with  respect 
to  left  eye.  The  cornea  segment  is  a smaller  sphere  that 
intersects  the  front  of  the  larger  sphere  to  produce  a more  more 
reallistc  eyeball,  right  eye  is  exactly  the  same  as  left_eye,  with 
£ Exception  th.t  ItVloc.fd  .t  (-3.2,  0,  0)  .1th  r..p.ct  to 
the  origin  of  the  model. 

VP's  description  language  is  similar  to  the  QuickModel  format 
developed  by  Alias  Research.  One  major  difference  is  that 
model  description  language  does  not  yet  support  spline  curves  and 
surfaces.  Another  difference  is  that  QuickModel  does  not  SUPP°^ 
hierarchical  model  descriptions,  while  VP  does.  The  model 
description  language  is  still  under  developnmnt-  efforts.re^being 
made  to  give  it  spline  curve  and  surface  support,  and  to  make  i 
more  compatible  with  the  QuickModel  format. 

The  specifies  a three-dimensional  graphical  model  is  defined  in 

vp/include/model  .h.  It  is  defined  as  type  Model  as  shown  below. 


typedef  struct  model  Model; 

struct  model  { . . 

char  *name;  /*  Unique  model  name.  */ 

Tree  treenode;  /*  Tree  node  that  contains  model.  / 
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Fla?a  status; 
Xfxm  local; 
Xfrm  global; 
Tree  segtree; 
*/ 


/*  Various  status  Flags.  */ 

/*  Coordinate  transfoaaation.  */ 
f*  Coordinate  transformation . *f 
/*  Segments  that  compose  the  smdsl. 


The  members  of  struct  model  are  briefly  described  below. 

name  — A unique  name  used  to  identify  the  model  The  string  in  the  — ~ Add  is  a 
concatenation  of  the  model’s  name  (as  specified  in  its  description  file)  and  a numerical  index 
that  is  assigned  in  a call  to  name_model. 

treenode  — A pointer  to  the  tree  node  in  which  this  model  is  stored 

status  — Various  status  flags.  At  this  time,  the  only  flag  muafly  toed  is 
status.ia_lit.  The  model  is  rendered  with  as  a solid  if  the  status. is  litflaxis 
true  at  the  time  the  model  is  being  drawn.  Otherwise,  it  is  drawn  as  wireframe.  ~ 


1®*al  ~ * ^ogeneous  transformation  matrix  that  specifies  the  location  and  orientation 
or  the  model  with  respect  to  its  immediate  predecessor  in  the  model  tree. 

global  — A homogeneous  transformation  matrix  that  specifies  the  model’s  location  and 
orientation  with  respect  to  world  origin. 

segtree  — The  tree  of  object  primitives  that  compose  the 

The  vertices  and  faces  of  the  model  are  actually  stored  in  the  tree  of  segments  specified  by  the 
segtree  field  of  the  Model  structure.  A segment  is  stored  in  a data  structure  of  type 
Segment,  which  is  defined  in  vp /include  /mode  l.h.  The  structure  is  shown  below. 

typedef  struct  segment  Segment; 
struct  segment  { 

/*  Segment  name  */ 

/*  Coordinate  transformation  */ 

/*  Coordinate  transformation  */ 

/*  when  drawing  as  wireframe 

/*  Number  of  polygons  in  the 

/*  Vertex  lists  for  each  faca  */ 

/*  Vertex  coordinates  */ 


/*  Number  of  vertices  in  the  face 
/*  List  of  the  face' s vertex 
/*  The  face's  surface  normal  */ 


char  *name; 

Xfrm  local; 

Xfrm  global; 
float  color [4] ; 

V 

int  nfaces; 

segment  */ 

Face  *facetable; 
float  *vertextable; 

} ; 

typedef  struct  face  Face; 
struct  face  { 

int  nvertices; 

*/ 

int  *vtable; 

indices  */ 
float  normal  [3]; 

) ; 
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The  members  of  struct  segment  sir  briefly  described  below. 

nmam The  of  the  segment  as  specified  in  the  model  description  file. 

local A homogeneous  transformation  matrix  that  tpMito  orientatioo 

of  the  segment  with  respect  to  its  immediate  predecessor  in  w segment  tree, 

global  — Not  yet  implemented,  this  field  wffl  be  the  homogeneous  transformation 
specifying  the  location  and  orientation  of  the  segment  with  respect  to  world  origin. 

color  [ 4] Specifies  the  color  of  the  segment  (Only  applies  when  die  segment  is  being 

drawn  as  a wireframe.) 

nf  aces  — The  number  of  faces  in  the  segment 

facetable  - A table  of  Face  structures.  Each  face  in  the  segment  has  a structure  that 
is  pointed  to  by  an  entry  in  facetable.  race  structures  specify  the  face  s aureus 
normal  and  the  list  of  vertices  that  make  up  the  face. 

vertextable  — A table  of  the  x,  y and  z coordinates  of  each  vertex  in  the  segment 
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APPENDIX  D 


— EYE  PHYSIOLOGY 
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EYE 


Lina®  visus 
Lens  crystalline 
Cornaa 

Camara  oculi  anterior  * 

Camera  oculi  posterior 

Sinus  venosus  sclerae  ' 

[Carved is  Schlemxni , Leuthij 

Zonula  ciliaris  [Zinru]  , \ y\ 

(spotia  zonular  ia)  \ \/ .1 
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Figure  1.  Aerodynamics/Guidance  Module  Configuration 


' ' PHASE  IV: 

AERODYNAMICS  / GUIDANCE  MODULE 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 

This  is  the  Software  Product  Specification  for  the  aerodynamics/guidance  module  of  the 
MIDAS  Software  System. 

1.2  SCOPE  OF  DOCUMENT 

This  document  is  primarily  focus^  on  i w devdSS  stege.  a^S™f^GM  remains 
module  (AGM)  developed  in  the  Pha  theorv  of  AGM  the  user  is  urged  to  refer 

the  same  as  that  of  Phase  III.  To  unders  oriemallv  developed  and  implemented  in 

aerodynamics  and  control  theory,  UNIX,  Fortran,  and  C. 

1.3  PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 

2.0  RELATED  DOCUMENTATION 

2.1  APPLICABLE  DOCUMENTS 

The  following  documents  are  referenced  herein  and  are  directly  applicable  to  this  volume: 


Field,  California  94035-1000,  June  1990. 

A Gessow,  G.  C.  Myers,  Jr.,  Aerodynamics  of  .he  Helicopter,  Federick  Ungar 
Publishing  Co.,  New  York,  December,  1952 

Center,  Moffett  Field,  California,  April,  1985 


Page  1-1 


^ Pr°eranlmine  L*W-.  Version  1.0, 

3.0  CONCEPT 

3.1  DEFINITION  OF  THE  AGM  MODULE 

The  Phase  IV  AGM  module  consisted  of  two  portions.  The  guidance  portion  confutes  the 
controls  required  to  steer  the  aircraft  to  the  desired  location.  The  aerE^sSon 

nnn,v,UtfS  the  Posltl.on  after  having  applied  the  controls  computed  by  the  guidance 
portion  for  certain  time  interval,  the  size  of  a tick  in  Phase  IV. 

3.1.1  Purpose  and  Scope 

The  Phase  IV  AGM  module  served  the  purpose  of  providing  a high  fidelity  aerodvnamW 

prl^LWh  guld,ance  capabilities  in  support  of  MIDAS  in  conducting  research  aStiSto 
address  human  factors  issues.  The  AMA-developed  AGM  is  linear and^vSSL? 
in  the  sense  that  the  collective  control  has  no  effecT ,n  the  yaw,  pitch,  orS  mwS^’ 

^ . e/[ect  ot! the  helicopter  behavior  was  not  taken  into  consideration  as  well  The 
model  has  been  enhanced  by  AMA  with  the  addition  of  a Tum-Straight-T^  guTdanc? 

previous  phases  A°M  W*S  POTted  t0  ^ A3l's  Silicon  GraPhics  Workstations  in 

3.1.2  Goals  and  Objectives 

DMA  terrain  with  the  necessary  features  specified  by  the  Symbolic  Operator  ModefnetSted 
to  be  rendered  using  the  techniques  developed  by  a former  A3I  graphics  staff  mem  her  in 

^omputeCdebSa|h'  11  hd3S 

Copilot,  Sndy ^ewS  mSe^0"’  “d  ^ ^ diSplayS  °f  the  VEST  Pilot-  ^ST 

3.1.3  Description 

^ AGM  t0  prov.id,e  Symbolic  Operator  Model  with  the  computed  controls  its 
structure  was  modified  from  closed  loop  to  open  loop  as  described  below  Fnrh  e,,~i 
begins  with  the  guidance  portion  computing  the  controls  based  on  the  current  position  and 

nnrHnn  hyt  does  not  pipe  the  computed  controls  directly  into  the  aerodynamics 

portion.  Instead,  the  controls  are  first  sent  across  the  network  to  the  Symbolic  Pilot  Model 

S* JS3*S  °I re,eC,S  ,he  “"'I01*  based  “P°”  his  availaW'  resourcl^SvmMc  ' 
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To  integrate  the  terrain  with  AGM,  the  AGM  module  was  enh^to  be  capable  of 
reading  die  terrain  data  and  storing  them  in  a proper  place  for  future  access. 

The  controls  actually  used  and  the  aerodynamic  parameters  were  sent  across  the  network  to 
Sve  thegraphics  displayed  by  the  VEST  Pilot,  VEST  Copilot,  and  Views. 

The  network  interfaces  required  for  AGM  to  interact  with  other  MIDAS  modules  were 
provided  by  the  Communication  module. 

3.2  USER  DEFINITION 

knowledge  accomplished  with  MIDAS  might  be  useful  to  users  who  attempt  to  accomplish 
similar  task.  The  capabilities  developed  could  also  aid  the  vision  model  in  performing  th 
line-of-sight  checking. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 

The  enhanced  AGM  is  capable  of  calculating  the  elevation  for  any  given  point  °f  *e  tOT^n. 

TtoSER  * wptod » ^meter-  ^ caPabi!ity  of  cb^.*e  Slght 

for  a^two  Points  can  be  used  in  assisting  the  pilot  in  carrying  out  his  mission.  The 
introduction  of  intermediate  waypoints  prevents  the  helicopter  from  flying  underground. 
The^ntermediate  waypoints  are  introduced  prior  tothecommencement  ofsimuiatton  based 
on  the  input  waypoints  specified  by  the  pilot  model  and  the  terrain  data.  When  the 
simulation  begins  the  helicopter  will  traverse  through  the  waypoints  contained  in  the 
expanded  waypoint  list . 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 

The  following  sample  operational  scenario  gives  the  reader  a flavor  of  how  to  run  the 
enhanced  AGM. 

The  input  required  by  the  AMA-developed  AGM  consists  of  a set  of  waypoints.  Each 
waypoint  cot  tains  not  only  the  coordinates  (x,y,z)  but  the  speed  and  hiding  of  die 
helicopter  at  that  point  as  well.  The  input  for  the  enhanced  AGM  requires  a sJl8hdy_ 
different  form.  The  user  needs  to  select  a set  of  two  dimensional I waypoints  (x,y)  from  the 
terrain  and  specifies  the  desired  altitude  above  terrain  at  each  of  these  points.  Also,  the 
desired  speed  and  heading  of  the  helicopter  at  each  of  the  selected  waypoints  need  to  be 

specified. 

n™  the  user  runs  the  AGM  module.  It  first  reads  the  user-specified  input  data.  Then  it 
calculates  the  terrain  elevation  for  each  of  the  selected  waypoints  and  adds  ton  the  altitude 
above  terrain  specified  at  this  point  to  yield  the  absolute  altitude.  The  line  of  sight  checking 
SSSSSt TJch  pair  of  consecutive  waypoints.  If  the  line oft sight for -a .pair  of 
consecutive  waypoints  is  not  clear  proper  intermediate  waypoints  will  be  introduced. 

SonS  sceeds  and  headings  need  to  be  specified  at  the  introduced  waypoints  as  well.  The 
w^in&Sned  in  til  new  list  will  become  the  ones  the  helicopter  has  to  traverse. 

The  initialization  process  always  positions  the  helicopter  at  the  very  first  waypoint.  The 
™fdan  ce pS  Itggests  to  til  Symbolic  Operator  Model  the  controls ..  computed  to  steer 
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S°Ptcr  t0^?.next  wayPoim.  The  Symbolic  Operator  Model  notifies  AGM  whether 
the  pilot  has  available  resources  to  perform  the  suggested  control  movements  If  the 
resources  arc  not  available,  controls  used  in  the  previous  cycle  will  be  used  in  the  current 

S?rnnS?  ac^y.namicj  portion  then  computes  the  new  position  and  orientation  basS  on 
Ae  controls  actually  used  Then  the  controls  and  aerodynamic  parameters  areTnt  ^ross 
the  network  to  dnve  the  displays.  This  process  continues  until  the  simulation  is  over. 

4.0  REQUIREMENTS 

4.1  REQUIREMENTS  APPROACH  AND  TRADEOFFS 

The  Phase  W AGM  module  needed  to  fulfill  the  following  software  requirements  First  it 
needed  to  be  able  to  read  the  data  of  any  DMA  terrain  andltore  th^raTSace7or 
easy  access.  Secondly,  efficient  schemes  needed  to  be  devised  (1)  to  interpolate  terrain 
elevation  at  a two  dimensional  point  based  on  the  known  elevations  at  v^es  (2)7o  check 
line  of  sight,  and  (3)  to  introduce  intermediate  waypoints.  Finally^canS  S for 
exchanging  information  with  other  MIDAS  modules  needed  be  provided 

4.2  EXTERNAL  INTERFACE  REQUIREMENTS 

The  Phase  IV  MIDAS  is  a truly  distributed  system.  Its  modules  are  distributed  over 

C<TpUter  P|!tforms-  To  allow  the  AGM  module  to  exchange  information  durine  a 
s mulation,  four  interfaces  need  to  be  developed  These  four  interfaces  allow  it  to  ® 

communicate  with  die  VEST  Pilot,  VEST  Copilot,  Views,  and  the  Sym£  (Cator 

^^oenwJhese  interfaces  ^ provided  by  the  Communication  module.  Data  structures  are 
designed  to  contain  the  controls  and  aerodynamic  parameters.  For  more  detailed 
information,  refer  to  Annex  J,  Communication  Module. 

4.3  REQUIREMENTS  SPECIFICATION 
4.3.1  Process  and  Data  Requirements 

TIk  input  waypoint  list  required  by  the  Phase  IV  AGM  module  was  described  in  section 

^riall 


4.3.2 


Performance  and  Quality  Engineering  Requirements 


’?  exPected  10  exhibit  reasonably  good  performance  so  that  it  can  fir  in 
e MIDAS  simulations.  The  computation  of  controls  and  the  integration  of  the  governing 

smu  a?oSn0Vpoia0hirrS,“  Sh°U-d  n0t  the  bottleneck  when  participating  in fthe  8 

simulation.  Portability  is  a major  consideration  during  the  development  stage*  The 

network c communication  between  the  AGM  module  and  other  MIDAS  modules  should  be 

Mos'  imponan"y’ 11  is  neCKS“?  ,0  S£$cta,b' 

4.3.3  Implementation  Constraints 

^e  AGM  moduie  was  developed  on  the  government-furnished  equipment,  which  includes 
the  Symbolics  and  Silicon  Graphics  IRIS  workstations.  Because  the  SGI  workstations 

scienp[,c  comPuting,  the  Fortran  AGM  module  was 
designated  to  run  on  the  SGI  workstations.  The  network  interfaces  available  on  the  SGI 
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workstations  are  written  entirely  in  C.  Therefore,  interfeces  are  needed  for  fee  Fortran 
AGM  module  to  communicate  with  the  C network  interface.  Fortunately,  the  SGI 
workstations  do  provide  ideal  tools  - the  data  blocks  and  mterlanguage  calls  to  implement 
the  C-Fortran  and  Fortran-C  interfaces. 

5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

Figure  1 shows  the  top  level  configuration  of  the  Phase  IV  AGM  ^^^^VFST^lot 
MED  AS  modules  it  directly  interacts  with,  such  as  the  Symbolics  Pilot  Model,  VEST  Pilot, 
VEST  Copilot,  and  Views.  Conceptually,  the  AGM  module  consists  of  four  major 
subcomponents  : terrain,  guidance,  aerodynamics,  and  network  interfaces. 


Figure  1.  Aerodynamics/Guidance  Module  Configuration 

The  network  interfaces  provided  by  the  Communication  module  were  implemented  based 
on  the  TCP/IP  protocol.  The  C-Fortran  and  Fortran-C  interfaces  are  used  in  facilitating  the 
communications  between  the  guidance/aerodynamics  and  fee  network  interfaces.  As 
mentioned  in  Annex  J,  Communication  Module,  the  AGM  module  communicates  with 
other  MEDAS  modules  through  the  communication  manager.  Data  structures  are  defined  to 
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m<^ulesC0ntr0lS  3,1(1  aerodynamic  Parameters  and  are  passed  around  among  the  MIDAS 

5.1.1  Design  Approach  and  Tradeoffs 

tom  sc,ralch'  b“»“  '•  only  needs 

sssssr iraprovin*  * agm  ■»**  *» ^ 

The  acquisition  of  a parallel  computer  can  change  the  entire  structure  of  MTn A <:  c™- 

»safK  sssss  on 

modules  running  on  the  parallel  computer  processors  hav?ac^«  to 

memoir  JaJSTo' *”  m“IU,eS  ‘°  S'0re/fach  d“  *e  global 

5.1.2  Architectural  Design  Description 

To  meet  the  requirements  stated  earlier,  the  architecture  of  the  Ar.iu i i ■ j 

movements.  The  third  subcomponent  is  the  network  inSS  M 

thC  C??1municatlons  with  the Symbolic PUot  Model 

mmmmm 

5.1.3  External  Interface  Design 
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5.2  DETAILED  DESIGN 

The  Phase  IV  AGM  software  was  designed  to  have  four  code  units.  These  four  cocte  units 
pSy withthe subcomponents1 described earlier.  Tbe code units are descnbed m 

detail  in  later  sections. 

5.2.1  Detailed  Design  Approach  and  Tradeoffs 

LTte  pS7ar0fnSSn":  nSfty  to  allow 

the  user  to  make  these  changes  easily. 

The  subcomoonents  of  the  AGM  module  were  implemented  in  a tmly  modular  fashion  and 
Ae  imerfaSramong  them  are  very  friendly.  Since  most  of  the  guidance  and  aerodynamics 
models  are  written  in  Fortran,  an  interface  was  implemented  forthe  ^Tmerface  was 
aerodynamics  models  to  interact  with  other  subcomponents  » C 3^rkItSons 
written  based  on  the  interlanguage  calls  and  dala  blocks  ava  lable  on  *e 
Refer  to  "IRIS-4D  Series  Fortran  Programming  Language  listed  in  section  . . 

Because  the  AGM  module  needs  to  communicate  wi*  the  communication  manager,  the 
network  connection  process  was  included  in  the  AGM  software. 

5.2.2  Detailed  Design  Description 

5.2.2.1  Compilation  Unit  Design  and  Traceability  to  Architectural  Design 

The  vertices  extracted  from  the  DMA  DTED  tapes  were  grouped  to  form  polygons^ 
Usudlya  polygon  has  three  vertices.  A polygon  may  have  four  vertices  * *ey  arc 
y'nnianar  Because  the  terrain  adopts  polygonal  representation,  a data  structure  is  defined 
forPa  polygon  in  a header  file  which  contains  the  number  of  vertices  and  the  coordinates  of 
each  vertex  An  array  of  this  data  structure  is  declared  in  die  main  function  that  holds  the 
information  of  polygons  that  make  up  the  entire  terrain.  The  array is  declared  to  be  globa 
so  that  it  is  accessible  to  both  the  guidance  and  aerodynamics  subcomponents. 

The  run-time  AGM  module  consists  of  the  following  processes  : initialization  process, 
connection  process  guidance  computation,  aerodynamics  computation,  data  passing.  The 
mainMCf funcdo^was^ wri  tten  which  reflects  the  control  flow  of  these  PJ*  ® 

executing  the  command  to  run  the  AGM  module,  the  communication  module  should  have 
proceeded  to  the  stage  ready  to  accept  requests  for  connection. 

The  initialization  process  does  the  following  things.  It  opens  the  file  that  contains  the 
of  terrain,  reads  the  data,  and 

data  structure.  It  then  reads  the  AGM  input  file  (see  AMA  Report  252-3  for  details)  and 
introduces  appropriate  intermediate  waypoints,  if  necessary. 

When  the  initialization  process  is  completed,  it  proceeds  to  the  network  connection  stage. 
At  this  point,  the  communication  manager  should  be  waiting  to  accept  connecuon  requests. 
The  AGM  module  sends  connection  request  to  the  commumcanon  manager.  When  the 
AGM  module  has  been  successfully  connected  to  the  communication  manager,  it  waits 
the  first  incoming  message  from  the  communication  manager. 
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^ p^2MT^^thenJStfr?  guidance-aerodynamics  cycle.  A new  important  capability 
the  Phase  IV  AGM  module  has  is  to  allow  the  pilot  to  change  the  route  of  flight  during  the  * 
simulation.  If  the  AGM  module  receives  a new  route  of  flight,  it  introduces  a new  set  of 
intermediate  waypoints.  The  new  waypoint  list  will  replace  the  old  list  and  the  helicopter 
now  has  to  traverse  the  new  set  of  waypoints.  At  the  end  of  each  cycle,  the  controls  and 
aerodynamic  parameters  are  sent  across  the  network  to  drive  the  graphics  modules. 

S.2.2.2  Detailed  Design  of  Compilation  Units 

gives  •he  **d‘r  1 fW  °f  •he 

TTie  elevation  of  a terrain  point  (x,  y)  was  calculated  as  follows.  First,  identify  the  polygon 
the  point  is  on.  TCien,  obtain  the  plane  equation  for  the  polygon  from  its  vertices.  Finally 
solve  for  z by  substituting  x and  y into  the  plane  equation.  ’ 

To  check  the  line  of  sight  for  two  given  points  (xl,  yl,  zl)  and  (x2,  y2,  z2),  first  project 

h;  connectinf  <xl’  X1*  z]>  *nd  (*2.  y2,  z2)  onto  the  horizontal  x-y  plane, 

^e  fonn  he  2d  line  segment  (xlyl)  and  (x2,  y2).  Identify  the  set  of  polygons  covered 
by  the  2d  line  segment.  The  line  of  sight  is  not  clear  if  any  polygon  in  the  set  lies  above  or 
intersects  the  3d  line  segment,  and  is  clear  if  the  polygons  in  the  set  all  lie  below  the  3d  line 

^*5  Here’  a hne  segment  is  said  to  intersect  a polygon  if  the  intersection  point  is  part 
or  the  line  segment  and  within  the  polygon.  r F 

The  algorithm  used  in  checking  the  line  of  sight  is  also  used  in  the  process  of  introducing 
intermediate  waypoints.  Suppose  we  are  given  a line  segment  defined  by  the  starting  point 

mvlri/iK  ' and  the  a^a  ?°mt  P2fx2,  z2)-  First>  identify  the  set  of  polygons 

covered  by  the  projected  2d  line  segment  (xl,  yl)  and  (x2,  y2).  Cycling  through  the 

polygons  in  the  set,  if  a polygon  intersects  the  3d  line  segment  (see  above  for  its 

£in“!°2’.an^tCrmedia!e  waypoint  ne.eds  ^ introduced.  The  introduced  waypoint  will 
be  the  intersection  point  elevated  a certain  height  in  the  z direction.  At  this  stage,  purge 

!SSp?Ha'faVe  5660  Processed  off  the  Poison  set  The  newly  introduced  * 
waypoint  and  P2  define  a new  line  segment  The  waypoint  introduction  process  for  the 

SfeTne  3 slfht*  ST"  ^ ^ ^ ^ i 

5. 2. 2.2.1  Detailed  Design  of  Initialization  Unit 

The  following  C functions  and  Fortran  subroutines  are  involved  in  this  unit* 
read_ter_data(),  openfilesj),  tdinput_(),  init_dp(),  and  get_dp_types(). 

?ri,iS?f/fi%CalteChnLqUe  availabif  on  **  SGI  workstations  is  used  in  this  unit  for  a 
C function  to  call  Fortran  subroutines.  Those  procedures  with  names  ending  with 

^ ,Aman  s“bro“:  Bas>cally.  read_ter_data()  reads  the  terfain  polygons. 
openfilesO  and  tdinputj)  read  the  input  data  and  invokes  functions  contained  in  chk  los  c 
waypoints.  Init_dp()  initializes  the  AGM-related  data  structures 
defined  in  the  data  pool.  Get_dp_types()  contains  the  message  type  and  the  source  and 
destination  modules  for  all  AGM-related  messages.  u uic  source  ana 

5.2. 2. 2.2  Detailed  Design  of  Connection  Unit 

is  ll?e  major  func»on  involved  in  this  unit  This  function  follows  the 
d ^ netwo.rk  connection  procedure.  It  involves  creating  sockets,  binding  port 
numbers  and/or  network  addresses  to  sockets,  and  sending  connection  request  to  the 
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communication  manager.  Hie  rocket  number  returned  from  successful  connect  call  is  used 
for  later  subsequent  data  transmission. 

5. 2.2.2. 3 Detailed  Design  of  Guidance  and  Aerodynamics  Units 
These  two  units  are  described  in  detail  in  the  AMA  Report  252-3. 

5.2.2.2.4  Detailed  Design  of  Data  Passing  Unit 

This  unit  contains  functions,  which  can  be  found  in  the  file  networks 
to  perform  the  following  tasks. 

. Update  the  related  data  structures  with  the  guidance-computed  controls  and 

. SSSrSffiST Sructures  across  the  nemo*  to  modules  that  need  them. 

* Detect  incoming  messages  on  the  socket. 

* Retrieve  incoming  messages  and  respond  to  the  . 

5.2.3  External  Interface  Detailed  Design 

The  external  interfaces  are  provided  by  Ore  Cmmmnuc.^ 

dt™^eSp^g«^sed  during  the  simulation.  Described  herein  are  these 
messages. 

■Dree  control-related  messages .are 

route  of  flight  and  the  aerodynamic  parameters.  See  p. 

5.2.4  Coding  and  Implementation  Notes 

The  compiler  oprions  used  included  -G  8.  -Olimit  1024,  -lm,  -Isnn.  -Ibsd,  and  -lc_s. 

6.0  USER’S  GUIDE 

6.1  OVERVIEW  OF  PURPOSE  AND  FUNCTIONS 

The  ACM  module  is  a Sparer Ta£S  us^h2odel  to 

accepted  by  the  aerodynamics  commu  y-  it  by  no  means  satisfies  missions  that 

require’ a^essive^aneuver^The  lack  of  the  nap-of-meretmh  capability  is  considered  a 
major  drawback.  It  also  lacks  the  terrain-following  capabdity. 

The  integration  of  <he  remain  into  the  ACM  modela 

c^bS^n‘SSgSa2 ihTvriioS  pilot  nrodels  to  aid  ihem  in  perfonning  .heir 
tasks. 


6.2  INSTALLATION  AND  INITIALIZATION 
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The  default  setup  designates  the  AGM  module  to  run  on  Starfish  (IRIS/4D  220).  Prior  to 
command  to  run  the  AGM  module,  the  communication  module  should  have 
progressed  to  the  point  ready  for  accepting  connection  requests.  When  the  message 

“jSSSjJ  ?cC  TOmmonic/0o°  010(1016  « ready  appears  on  the  screen  of  the  workstation  on 
which  it  runs,  issue  the  AGM  command  run_agm".  And  this  is  all  you  need  to  do'  If  the 

p?or  10  ±e  point  **  communication  module  is  ready, 
themcxlulewill  exit  after  a number  of  attempts  to  connect  to  the  network  server.  The  AGM 
module  will  proceed  in  the  way  described  earlier. 

6.3  STARTUP  AND  TERMINATION 

The  required  input  data  are  clearly  described  in  AMA  Report  252-3.  Once  this  file  is 
prepared,  issuing  the  command  "run_agm"  on  the  designated  workstation  is  all  you  need  to 
do  to  run  AGM.  When  the  simulation  has  progressed  to  the  end  of  the  duration  specified 
by  the  user,  the  simulation  executive  notifies  all  modules  of  the  termination.  ^ 

No  special  procedures  are  provided  when  it  encounters  abnormal  conditions  In  such 
cases,  just  re-run  the  simulation. 

6.4  ERROR  AND  WARNING  MESSAGES 

£^nCth^ ir°M  incluclf d in  connection  unit  to  ensure  successful  connection 

between  the  AGM  module  and  the  communication  module.  Also,  the  waypoints  are  also 

a^an.  a6a‘nS'  ^ b°l,n<W'S'  " My  Wayp0im  “ °>“  <*  W. 

6.5  RECOVERY  STEPS 

It  is  suggested  that  die  user  follow  the  User's  Guide  section  to  restart  the  AGM  module  if 
any  abnormal  situation  occurs. 

7.0  NOTES 

7.1  LESSONS  LEARNED 

During  the  development  stage  of  the  Phase  IV  AGM  module,  the  developer  learned  that  the 
integration  of  the  terrain  with  the  AGM  model  paved  the  way  for  future  development  of  the 
terrain-following  capability.  It  needs  to  be  determined  whether  the  model  currently  used 
"f?1  *e  ^ssion  requirements  in  future  phases,  as  the  pilot  model  developer  has 
two^  phases  °f  ^ Inadequacy’  relative  to  the  mission  complexity,  of  the  current  model  for 

7.2  FUTURE  DIRECTIONS 

E’iKSS8  m<*Ju,e. ' the  following  things  might  be  worth  pursuing.  It  needs  to 

be  detemuned  whether  to  develop  desired  capabilities  such  as  NOE  and  terrain-following 
capabiliucs  or  locate  existing  models  which  already  have  these  capabilities.  The  modd  with 
NOE  capability  used  by  another  task  at  Ames  sounds  attractive  and  is  worth  exploration. 
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' PHASE  IV: 

COMMUNICATIONS  MODULE 


1.0  INTRODUCTION 

1.1  IDENTIFICATION  OF  DOCUMENT 

This  is  the  Software  Product  Specification  for  die  communication  module  of  the  MIDAS 
Software  System. 


1.2  SCOPE  OF  DOCUMENT 

This  document  describes  the  **£«%££*  f'SnShXi*  fe  fc'and  WK 
communication  module.  The  re  laneuaees  TCPAP  protocol,  and  basic  concept  of 

Sf  development  is  definitely 

helpful,  but  not  required. 


1.3 


PURPOSE  AND  OBJECTIVES  OF  DOCUMENT 


rtivrv/ju  

The  purpose  of  this  document  is  to  ^^“^“tonOT^uo^^U^'evef  information 
of  MIDAS  modules  ach,ev«l  .» pta * »•»*«;  communication  module.  It 

SSSte  imptoSon  detail  useful  to  users  involved  in  modifying  the 
mmmunication  software  to  meet  their  needs. 


2.0  RELATED  DOCUMENTATION 

2.1  APPLICABLE  DOCUMENTS 

The  following  documents  am  referenced  herein  and  are  direcdy  applicable  to  this  volume: 

Silicon  Graphics  Inc.,  HUS-4D  Series  Communicadons”.  Version  1.0,  Mountatn  View, 
California,  1988 

Symbolics  Genera  7.*  Manual,  Vol.  5,  "Streams,  Files,  and  I/O 
Symbolics  Genera  7.*  Manual,  Vol.  9,  "Networks" 

2.2  INFORMATION  DOCUMENTS 
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3.0  CONCEPT 


3.1 


DEFINITION  OF  THE  COMMUNICATION  MODULE 


The  phase  IV  communication  module  was  defined  to  be  the  one  responsible  for  nrovirfin* 

'SfSSSK'  mssaec  passing  b“w"n  “"’AS  "»**• 

3.1.1  Purpose  and  Scope 

TTie  Phase  IV  development  of  MIDAS  focused  on  the  integration  of  modules  so  that 
simulations  could  be  earned  out  to  show  quantitative  (loading  and  timelines)  and  aualitariw 
(equipment  design  characteristics)  differences  between  the 

nha«<*C|vlin0n»SCi  of.mis^<^  tasks.  The  purpose  the  communication  module  served  iif 
phase  IV  was  to  devise  viable  schemes  to  integrate  various  model-based  MIDAS 
and  ensure  successful  simulation.  The  schernTdevS^^^^ 

&”.£££rl te' ***' 101 Khi'vei 

3.1.2  Goals  and  Objectives 

The  goal  of  the  phase  IV  communication  module  was  to  develoo  a fbnvwnrt  f™-  i~,„„ 
scale  distnbuted  system  integration  so  that  MIDAS  modules  coiSd  teimeeratedTn^ 

co“ld  Performed  to  address  important  human  factors  research  problems 
The  framework  is  intended  to  yield  a modular  environment  that  ease«  hnth  tlwacir  nt 

m,das  ■«— - ®^ss£tJsiss, 

3.1,3  Description 

MmtAclly’^,e,COmmUilication  module  was  designed,  as  shown  in  Figure  1 to  network 
nL^EA,S  m°dules  together  so  that  messages  can  be  passed  around  betwSS  them  * 

sidesends°  ^ “ h“  10  ass“rc  ““  side^ve.  p^XlTsending 
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Fig.  1 

Figure  1.  Communication  Module  Overview 

Because  of  the  way  the  Symbohcs  mc^u^  Operator 

documentation  for  the  Scheduler  ^ddl^a^.  , interacted  with  the  Communication 
Model  was  the  only  Symbolics  toS^oXwith  the  Symbolic 

module.  The  othe^ymbolics  m^u  g module  needed  to  provide  sufficient  links, 

system. 

3.2  USER  DEFINITION 

The  configuration  and  tools  devel°t?^ * " -bas^^ules^istributed  over 

perform  large  scale  system  Integra  communication  applications  software  to 

MIDAS  system  need  not  reinvent  the  Wheel. 

3.3  CAPABILITIES  AND  CHARACTERISTICS 
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Genera7??i W,0rkstatlon  and  a Symbolics  workstation.  This  module  runs  under 
SSSmBSD? ? Under GKenera  8 0)  and UNIX System  V with  Berkeley 

or Genera cTmTS, 1,6  ST*?  for  the  modu,e  to  ™ under  different 

user  *>  5 as z *m 

3.4  SAMPLE  OPERATIONAL  SCENARIOS 

The^fbUowxns  sample  operational  scenario  gives  the  reader  some  flavor  how  a simulation 

Prior  to  running  a simulation,  the  user  selects  the  participating  modules  and  dedans  , 
appropriate  workstation  for  each  of  them  to  run  on  by  editing^  text  fife  Aki  Z r / 

t S3  S' is  lhe  duration  of  ,he  simula,‘o" in  -S  £ J L™  boSTS? 

file  and  the  communication  module  reside  on  the  same  workstation  TRKJniin™ 
Suppose  the  Symbolic  Operator  Model  and  the  AeSSSSl  wSe  *1^7' 

fiuS4D ”d  Were  des,gnaled  lo  ™ Symbolics  3675  and  the 

The  user  now  brings  up  the  communication  module.  It  first  reads  the  text  flip  to  ct  »h» 

eSSinp^f, rSimuIatlon  and  the  participating  modules.  Then,  the  connection  pfocess  of 
establishing  necessaiy  communication  channels  is  initiated.  In  phase  IV  onlv  one  stream 

was  used  in  each  channel  for  message  passing  between  the  communication  module  and 
each  of  the  participating  modules.  The  communication 

status  ready  for  accepting  requests  for  connection  from  the  participating  modules  While  it 

rr  ^ngS  UP  the  P^dcipating  modules  on  CSft^Sstato 
The  first  thing  they  do  is  issue  request  for  connection  to  the  communication 3, £ ^ 

connected”  PTOCeSS  continues  until  a11  participating  modules  have  been  successfully  ‘ 

samp^aSS^ 

wayjxnms  rathe  Aero/Guidance  model;  each  waypoint  conlins  MeSS 

TTie  simulation  scenario  gets  updated  from  this  point  on.  Each  participating  module 

endofStheSt[ck  WhelTS  tick  UpdateS  ‘hC  aSSOcia,ed  d*a  structures  at  the 

data  structured  The  ,°Ck  ^ ”s’ the  PartlclPating  modules  exchange  the  updated 

tciflS  Se  cominues  ,n  ,his  wa>’ for  as  Iid“  “ »*  «? 
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4.0  REQUIREMENTS 

4 1 REQUIREMENTS  APPROACH  AND  TRADEOFFS 

■ mrviiilp  needed  to  fulfill  the  following  software  requirements. 
The  phase  IV  communication  ™^  nenfefwhich  ensured  that  all  participating  modules 
First,  it  needed  to  serve  as  the  control  cente  expected  to  function  as  the 

'teSS  existing  MIDAS  modules  distributed  over  dtiferen, 

platforms. 


need  to  oe  aevciu^,  . • . rface  is  to  allow  an  IRIS  moauie  10  exchange 

as  described  in  section  3.1.3.  ld  The  second  interface  is  to  allow  a Symbolics 

information  with  the  rest  of  the  ’ ^ symbolics  world.  The  third  interface 

module  to  exchange  information  world  so  that  every  module  can  exchange 

is  to  link  the  Sfai  requirement,  for  the  external 

jSSSESSS  ^ " ,0  dc,Kt  incomi"8 

4.3  REQUIREMENTS  SPECIFICATION 

4.3.1  Process  and  Data  Requirements 

■ , aHnntpH  a verv  simple  way  for  the  user  to  specify  the 
The  phase  IV  communication  modu  P required  input  data  include  the 

input  data  necessary  to  run  a MIDA^  S ™ h participating  modules,  and  the  workstations 

."d  S/5  acldexmthesanw'purpose. 

4.3.2  Performance  and  Quality  Engineering  Requirements 

The  communication  module  speed  is  required  to  be 

development  stage. 

4.3.3  Implementation  Constraints 

All  MIDAS  modules  were  developed  °"  Because 

*■—  teed  ,h‘ 

network-related  functions  available  on  these  workstations. 
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5.0  DESIGN 

5.1  ARCHITECTURAL  DESIGN 

As  mentioned  earlier,  the  phase  IV  MIDAS  system  can  be  thought  of  as  comno^d  nf  th* 

SS°  CS  world  “d  <he  ms  world.  Figure  2 shows  in  detail  the  phase  d MTOAS 
onfiguranon  and  the  decomposition  of  the  communication  module  uito  subcomnonents 
JJ^ymbo^sworld  consisted  of  the  Operator  Model,  Equ.Wnt^ 

Model,  and  Scheduler.  The  IRIS  world  displayed  the  VEST  Pilot  View  VFST  rvmiw^ 
Vtew,  and  World  View  Because  the  IRIS  woLtations  pmvS?tet^\™ 
environment,  the  Aero/Guidance  Model  (AGM)  is  included  in  the  IRIS  w° Sd 

tSSSEL  253?-  “ “ •-•SSSCiSS."-  s-! 
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Figure  2.  MIDAS  Phase  V Communication  Architecture 
The  Symbolics  modules  are  distributed 

Mrol^S^OTl^tad^ns  ^^neworki^  over  EAernet. 

^^^^aSl^^S^S^°^^haos  tose^^s^eCpro^d^w  netwoA  th^S^m^lkSt  ^ 

^^rat^MSel^^h^e  'rwt  oCS^ntwlics  modufe.  The  Symbolics  and  IRIS  worlds  are 
networked  by  the  TCP/IP-based  I-S  and  S-I  data  transceivers. 


Page  J-7 


represents  the  standard  procedure  a simulation  goes  through. 

5.1.1  Design  Approach  and  Tradeoffs 

driven  approach,  which  conh/nssul.  £,«’ T 

requmng  more  complex  coordination  method.  P or  me  simulation  at  the  cost  of 

During  a simulation,  the  MIDAS  modules,  distributed  over  different  comnnw 

k! tl0nS>  !lrave  to  exchanSe  information  frequently.  Some  strategies  were 
yield  better  performance,  not  necessarily  real-tirne  but^t  least  inactive  adopted  to 

ructure  as  a whole  between  a Symbolics  workstation  anH  on  tdto  i • ^ 

5.1.2  Architectural  Design  Description 

SSSSSSSSl ttS&PZSL  *• 

communication  manager  before  they  reach  desrifatin^m^f 3geAhaVC  to  g°  throu8h  the 
between  the  IRIS  modules  asTot  aJlowS  Therefore  STS!!?™!  ““«?  P3SSing 
to  other  modules,  all  it  needs  to  do  is  to  th * 9 a mc^u^e  nee^s  to  send  a message 

— £dir^ 

pool, residing ,he BUsSSJS. tl^TtTre* “T  a 

viabSjr  'ZuMot' "fScsS1'  T*  »f  “^n™Se 

pool  for  the  values  of  state  variables  thtough  the  commnnSon  da“ 

5,1.3  External  Interface  Design 
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message  which  signals .the  boty"  The  meslige  id  is 

messages  have  a specific  data  st™cture  assoc  message  is  detected,  the 

depending  on  the  message  id. 

5.2  DETAILED  DESIGN 

As  mentioned  earlier, 
tick  size  set  to  one  u' 


9 

tick  size  set  to  one  hundred  mil^eronds^  MIDAS  to  digger  individual  processes. 

simulation  executive  of  the  completion  for  the  cuirent  tick. 

There  were  approximately  two  ^hundred repreS^^^°a^roximately 
modules.  These  state  vanabkswere  g y ^uivalent  representations  were  defined  in 
SCSfiJSSSS  ^gSSLX  — & dinned  in  the  phase  IV 

MIDAS. 

The  task  of  providing  capabilities  capabilities  to 

modules  was  divided  into three  su  ■ , , within  the  Symbolics  world,  (2)  between 

Se  Ses  wiE'th!  IRIS  wo5ld,  and  (3)  between  a Symbolics  module  and  an  IRIS 

module. 

h"bTonTCT$$^ 

based  on  TCF/ir  proioeoi.  11  . . mTc  woriH  and  is  responsible  for 

The  I-S  data  transceiver  is  residentinthelR  'J  ld  s-I  data  transceiver  is 

between  these  two  worlds* 

5.2.1  Detailed  Design  Approach  and  Tradeoffs 

Byte  orders  are  preserved  when  mw^^tdartaare^>assed  between  a 

same  world.  However,  by^s.le* 1 , In  phase  iv,  the  S-I  data  transceiver  assumed  the 
Symbolics  module  and  an  IRIS  module.  * the  numcric  data  transmitted  to  or 

responsibility  of  performing  Pr0^J  ^ transmitting  numeric  data  across  the  Ethernet,  the 
received  from  an  IRIS  module  E «fore  t Stiver  jus.  collects  die  by.es 
S-I  data  transceiver  re-positions  them  so  that  th  . . numeric  data  from  the 

in  the  order  received  and  forms  numenc  da*  Aftcr  mcemng  ^ ^ forms 

Ethernet,  the  S-I  data  XT^trSvert  ^ « superior  to  the  I-S  data 
kanSrePta%rflung  b%  operations.  The  I-S  dan.  mmsceiver  can  do  the  job  as  well. 

Another  soategy  was  adopted  mr^etwo^^  nS ^ 

communication  manager  is  the  message  r g during  a simulation,  if  a data 

query  the  data  pool  for  the  data  they  needed.  At .any  stag®  ™™g  updated  data 

2tru2ure  gets  updated  by  a module,  Sen  updates  the 

Sapool  and  forwards  i,  mdesdnadon  modules. 
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5.2.2  Detailed  Design  Description 

5.2.2.1  Compilation  Unit  Design  and  Traceability  to  Architectural  Design 

Figure  3 shows  the  control  flow  of  the  communication  module.  It  also  represents  how  a 
simulation  proceeds.  In  particular,  it  maps  well  to  the  sequence  of  the  run-time  simulation 
processes  : initialization  process,  connection  process,  and  data  passing. 


Fig.  3 

Figure  3.  Control  Flow  of  the  Communication  Module 
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S.2.2.2  Detailed  Design  of  Compilation  Units 

As  mentioned  earlier,  prior  to  ning  a ™ * 
die  simulation  in  terms  of £mn  simulation  was 

Each  message  was  identified  by  an  -BJ^^SSSil,, 
defined;  these  data  structures  represent  the  gro  P , . . deader  file  dp.h,  which  was 
message  ids  and  the  set  of  data  t^SSTuch  as' VEST  Pilot, 

VK  “Stot  an%“r“Xlow  ant  the  J software  units  that  actually  perform  the 
mn  simulation  as  described  in  the  architectural  destgn  section. 

5 2 2 2 1 Detailed  Design  of  Initialization  Unit 

An  array  of  this  data  structures  was  dec  . . strucsO  to  initialize  this  array 

panic, pat, ng  modules.  The  mat”  dpToet.procsO 

modules  for  each  data  structure. 

5. 2. 2.2.2  Detailed  Design  of  Connection  Unit 

Establish_networkO  is  the  major 

standard  procedure  to  bring  the  commumcanon^g^fteaamm  fcy  ^ ^ 

connection  requests  from  other  module  , listening  for  and  accepting  connection 

numbets  and/or  network  addresse  to: s«keR^«mj|  ™ = P t0  g* 

STniS^^ 

stored  for  use  in  subsequent  data  transmission. 

5.2.2.2.3  Detailed  Design  of  Data  Passing  Unit 

SS'c  s£? 

iXd"Sr messale  bSdy,  depending  on  the 

5.2.3  External  Interface  Detailed  Design 

The  message  id  is  represented  by  an  ‘"“f^^K^aSsSMhe 
K^intg“em«Tag?KK  4 the  messagi  body,  depending  on  the  message  id. 
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5.2.4  Coding  and  Implementation  Notes 

The  compiler  options  used  included  -Isun,  -Ibsd,  and  -lc_s. 
6 0 USER'S  GUIDE 


6.1 


OVERVIEW  OF  PURPOSE  AND  FUNCTIONS 


^^©ASm^ufe^^ich6  arc  vSttwi'inlusp11  C ^dV^ t0  dTJ°P  ■ suitc  to  “te«rate 
IRIS  and  Symbolics  workstations  Fortran  and  distributed  over  the 

and  pnividi  ■fcS.’SSSB  £ SSK"  SOftWarc  « «*»  h c* IS Lisp 

6.2  INSTALLATION  AND  INITIALIZATION 

«« «"« »p  ^ 

screen  indicating  it  is  ready  to  accwcomretSn  ™ '?herf  a™essagc  appears  on  the 
up  the  participating  modules.  Otherwise  the  oartiemarf™  A<!JhiS  moment  the  user  brings 

sssar*" ,o  cMnec' was 

Starfish  '3?1'.  VEST  Copilot  on 

the  Symbolic  Pilot  Model  on  Barracuda  (3tf75)  IRIS)’  Aero/Guidance  on  Starfish,  and 

Pilot  wifi  peribnmi^ne^  -I^TA^bcoralsim"  on  Coral.  VEST 

the  CDE  item  from  the  MultiSn  menu  bar  drae  he  SCtUpc1idone* mousc  click 

the  mouse.  The  VEST  Pilot  will  uy  to  connea  to  the  rnmm  d t0  EtheJrnet.  and  release 
connection  is  successful  a 1 the  communication  module.  If  the 

DhAT°ATUniati°n  module  rons-To  bring  up  ^S  SflotZ5e*-W^Stetion  °n  which 

DATA/lbstarfsim".  To  brine  ud  Views^!P  / [ ,P  L°‘'  mgflt.starf  - 
Copilot  and  Views  follow  the  same  procedure  to!^nI^^’DATA^.,TC!,insi,n"  VEST 

wSi?s?! m0odu'SpT^°KtC‘  T° 

A 1 CT  AnTim  ^ 


6.3  STARTUP  AND  TERMINATION 


of  simulation  in  term^of  tick!  thTpanicTpating  mtSs^ co"tains  1,16  duration 
modules  run.  Once  this  file  is  ready,  follow^he  fren^fc  w !hetwoirkstations  these 
section  to  start  the  simulation.  The  simulation  will  in  the  User’s  Guide 

duration  specified  in  the  input  file.  terminate  when  it  has  gone  through  the 

«««.  in 

the  participating  modules.  8 °Wn  the  commumcation  module  posterior  to 
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6.4  ERROR  AND  WARNING  MESSAGES 

Certain  check  points  were  included  in  the  connection  unit  to  make  sure  all  participating 
SSes  had  been  connected  successfully  to  the  communication 

points  were  also  inserted  to  make  sure  the  communication  module  reads  enough  bytes  from 
the  sockets. 

6.5  RECOVERY  STEPS 

It  is  suggested  that  the  user  follow  the  User's  Guide  section  to  restart  the  simulation  if  any 
abnormal  situation  occurs. 

7.0  NOTES 

7.1  LESSONS  LEARNED 

During  the  development  stage  of  the  phase  IV  communication  module  *e  ^veloper 
learned  that  the  task  of  integration  is  a horrendous  and  challenging  job.  It  requires 
StefSaSSSS  among  the  module  developers.  And  the  individual  modules  ought  to 
be  well  developed  before  the  attempt  of  integration.  Hope  that  this  will  be  improved  in 
future  phases  if  integration  remains  as  a focal  point  ot  M1DA5. 

The  overall  performance  in  phase  IV  required  about  a second  clock  time  to  process  a tick. 
Seven  integrated  modules,  networks,  graphics,  and  computation  conmbuted  to  thi 
performance.  Networks  and  graphics  were  probably  the  main  contributors. 

7.2  FUTURE  DIRECTIONS 

To  improve  the  MIDAS  performance,  the  following  things  might  be  wor*  pursuing. 
Explore  alternatives  to  transmit  numeric  data  between  a Symbolics i workstation  and  an  MS 
workstation  In  particular,  it  would  be  nice  if  data  structures  could  be  passed  around  as  a 
whole  between  aPSymbolics  module  and  an  IRIS  module.  It  is  worthwhile  exploring  oth 
communication  protocols  with  less  overhead.  A parallel  computer  could  even  totally 
eliminate  the  network  bottleneck.  Reduce  the  graphics  overhead  embedded  in  MulnGen  by 
developing  a package  for  MIDAS  purpose. 
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